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Foreword 


Access  to  Information  Is  an  ever-increasing  concern  throughout  the  Air 
Force.  Large,  centralized  computer  systems  have  been  with  us  for  many 
years.  The  growing  number  of  personal  computers  available  to  Air  Force 
organizations  brought  with  it  a  growing  desire  for  information  that  is 
accurate,  timely,  and  in  the  proper  format.  Also,  there  has  been  a  cor¬ 
responding  growth  in  the  computer  literacy  level  of  Air  Force  personnel  that 
resulted  in  the  growth  of  highly  specialized  information  systems. 

MaJ  Charles  A.  Hobbs’s  research  builds  on  existing  knowledge  of  infor¬ 
mation  systems  by  explaining  the  differences  between  centralized  and 
decentralized  computing,  and  the  strengths  and  weaknesses  of  each.  These 
distinctions  provided  the  basis  for  a  survey  of  Air  Force  Audit  Agency  (AFAA) 
personnel  which  determined  user  information  requirements  that  recom¬ 
mended  a  combination  of  centralized  and  decentralized  information  sys¬ 
tems. 

Although  Major  Hobbs’s  report  addresses  only  the  specific  needs  of  the 
AFAA,  his  premise  that  modem  information  systems  should  consider  the 
desires  of  the  end  users  and  should  combine  thp-best  attributes  of  central¬ 
ized  and  decentralized  systems  applies  to  any/organlzat ion .  r 
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Director 
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Preface 


In  June  1988  John  Boddie.  auditor  general  of  the  Air  Force,  tasked  me 
to  assess  the  Information  system  requirements  of  the  Air  Force  Audit  Agency 
(AFAA)  as  a  research  fellow  for  Airpower  Research  Institute  (ARI)  at  the  Air 
University  Center  for  Aerospace  Doctrine,  Research,  and  Education 
(AU CADRE).  Maxwell  AFB,  Alabama.  Those  familiar  with  ARI’s  reputation 
as  “the  doctrinal  think  tank  of  the  Air  Force"  may  wonder  about  the 
relationship  between  air  power  doctrine  and  information  systems.  Al¬ 
though  this  research  does  not  provide  new  Air  Force  doctrine,  it  gave  me 
an  increased  appreciation  for  the  relationship  of  support  agencies  (such  as 
the  AFAA)  to  the  operational  mission  of  the  Air  Force.  Perhaps  more  than 
any  other  factor,  this  appreciation  was  my  most  lasting  legacy  from  this 
assignment. 

There  are  many  people  to  whom  I  owe  thanks  for  their  contributions  to 
this  project.  John  Boddie  provided  the  challenge.  Brig  Gen  Basil  Pflumm 
and  Kenneth  Seifert  gave  me  additional  guidance  and  direction.  I  received 
immeasurable  assistance  from  Troy  Wideman’s  staff  at  AFAA  headquarters. 
Jerry  Klee  man.  Don  Locke,  and  Ken  Dobbs  all  spent  considerable  time 
assisting  me  on  the  project;  and  Shirley  Wilson  provided  the  administrative 
support  necessary  to  distribute,  collect,  and  account  for  over  900  lengthy 
surveys.  Richard  Efnor  supplied  critical  information  on  AFAA  computer 
systems.  I  owe  special  thanks  to  CADRE’S  staff.  Lula  Barnes  and  Sheila 
Fears  of  AU  Press  Inputted  thousands  of  survey  responses  into  a  data  base 
for  analysis.  Dr  David  Maclsaac  provided  me  with  several  useful  articles 
on  information  system  design.  Dr  Bynum  Weathers  (research  chair)  and 
Dianne  Parrish  (research  editor)  gave  me  the  benefit  of  their  wisdom  and 
experience  in  the  field  of  academic  research.  Lt  Col  Manfred  Koczur  was 
always  there  with  an  encouraging  word  when  things  looked  bleak.  Finally, 
I  owe  the  deepest  debt  of  gratitude  to  my  wife  Nancy  and  my  sons  Al.  Tom. 
and  Ross  who  put  up  with  my  long  absences  and  short  temper  during  this 
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Chapter  1 

Introduction 


Many  of  us  worship  machinery  as  fervently  as  our  primitive  ancestors  worshiped  sucks 
and  stones. 

Neil  Macdonald 

12  Aphorisms  Regarding  Thinking 

Military  personnel  have  come  to  worship  the  computer  as  the  machine  of 
the  future  and  there  Is  ample  Justification  for  awe  of  modem  computing 
power.  An  ever-growing  number  of  personnel  have  computers  at  home,  at 
work,  or  at  both  places.  The  computer  age  has  arrived  throughout  the 
Department  of  Defense  and  has  increased  mission  effectiveness  in  many 
diverse  areas.  For  example,  the  US  Army  has  integrated  computers  down 
to  unit  level  to  field  and  support  a  more  combat-ready  force. 1  The  Air  Force 
also  uses  computers  for  a  variety  of  applications  to  include  militaiy  and 
civilian  payrolls — even  for  reserving  rooms  for  military  travelers.2  Presently, 
the  personal  computer  is  a  common  fixture  in  most  offices  in  the  Air  Force. 

As  users  become  Increasingly  familiar  with  the  functions  and  capabilities 
of  modem  computers,  they  need  not  “worship  the  machine.”  A  computer 
possesses  no  magical  capability  to  solve  an  organization’s  problems.  An 
organization  should  establish  its  automation  requirements  based  on  opera¬ 
tional  needs — mission  and  special  requirements.  Thus,  they  must  remem¬ 
ber  that  the  value  of  the  computer  lies  solely  in  the  increased  capabilities 
it  provides  to  the  user. 

Unfortunately,  users  often  obtain  the  latest,  state-of-the-art  computer 
equipment  and  software  without  considering  its  usefulness  to  the  organiza¬ 
tion.  Those  who  doubt  this  tendency  should  look  at  the  aggressive  market¬ 
ing  campaign  of  computer  companies  to  sell  color  printing  capabilities  for 
desktop  computers.  For  example,  users  have  paid  as  much  as  $75,000  for 
this  color  capability,  and  for  some  it  was  a  wise  investment  Other  users 
may  have  purchased  a  capability  for  which  they  have  little  use.  As  one 
dissatisfied  user  put  it,  “I  suppose  I  could  print  out  letters  in  different  colors, 
but  that  doesn't  really  Justify  the  cost.  3 

My  purpose  in  writing  this  report  is  twofold.  First,  Joui  Boddie,  the 
auditor  general  of  the  Air  Force,  tasked  me  to  identify  the  requirements  for 
a  management  information  system  (MIS)  for  the  Air  Force  Audit  Agency 
(AFAA).  Since  the  AFAA  is  the  sponsoring  agency  for  my  study,  this  purpose 
remains  paramount. 


My  second  purpose  evolved  as  I  researched  the  literature  on  systems 
analysis  and  design.  The  need  to  base  computer  acquisitions  on  desired 
capabilities  (rather  than  desired  hardware  or  software)  had  little  coverage 
in  the  literature.  Perhaps  this  lack  of  discussion  is  because  acquiring 
computer  systems  based  on  requirements,  rather  than  personal  preference, 
seems  like  an  obvious  consideration.  However,  my  experience  as  an 
information  systems  auditor  with  AFAA  has  taught  me  that  computer  users 
often  buy  systems  based  on  personal  preferences.  Therefore.  I  am  writing 
this  report  in  hopes  that  it  may  serve  as  a  guide  for  others  in  defining  their 
computer  needs. 

This  report  consists  of  six  chapters.  This  first  chapter  provides  an 
introduction  to  a  management  information  system.  Chapter  2  provides  a 
working  definition  of  a  management  information  system,  a  term  that  seems 
to  have  as  many  definitions  as  there  are  potential  users.  Further,  the 
chapter  discusses  the  system  development  process.  Finally,  and  most 
important,  chapter  2  explains  the  need  for  user  involvement  in  identifying 
the  specific  requirements  of  the  MIS. 

Chapter  3  describes  the  mission  and  organization  of  the  AFAA.  As  the 
internal  auditors  for  the  secretary  of  the  Air  Force,  the  AFAA  has  a  unique 
mission  and  unique  information  requirements  because  it  interfaces  with 
many  government  organizations.  Thus,  the  background  information  in  this 
chapter  will  help  you  understand  the  specific  reporting  requirements 
discussed  in  the  remaining  chapters. 

Chapter  4  lists  the  current  suite  of  computer  hardware  and  software 
within  the  agency.  The  auditor  general  has  suggested  that  any  future 
automated  system  should  use  existing  hardware  and  software  to  the 
maximum  extent  possible.  Thus,  the  data  in  this  chapter  is  for  planners 
who  will  design  the  system. 

Chapter  5  specifically  identifies  AFAA  information  requirements  through 
a  survey  of  personnel  at  all  levels  of  the  organization — from  administrative 
staff  to  senior-ranking  executives.  Additionally,  this  chapter  provides  an 
assessment  of  how  well  current  management  methods  are  meeting  the 
organization’s  Information  requirements. 

Finally,  chapter  6  provides  the  auditor  general  with  the  “next  steps" 
needed  to  transform  the  information  requirements  of  AFAA  users  into  a 
workable  format  (MIS).  Chapter  2  takes  the  first  step  in  this  process  by 
defining  a  management  information  system. 


Notes 

1 .  li  Gen  David  Doyle.  “Information  as  a  Hammer  In  Forging  the  Ready  Force."  Army. 
October  1985,  270-75. 

2.  Capt  James  Cox.  “SIMS — The  Computers  Are  Coming."  Air  Force  Engineering  and 
Services  Quarterly,  Fall  1984.  25. 

3.  Suzanne  Welxel,  “But  Do  You  Want  Color  on  Your  Desk?"  Computerworid,  9  May 
1988.  58. 
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Chapter  2 


The  Design  of  Management 
Information  Systems 

Because  management  information  systems  are  most  commonly  imple¬ 
mented  using  computers,  the  design  of  an  information  system  must  be 
based  on  a  knowledge  of  available  computer  hardware.  Basically,  informa¬ 
tion  systems  can  be  either  centralized  or  decentralized.  Centralized  sys¬ 
tems  are  run  on  large  mainframe  computers  and  decentralized  systems  are 
run  on  a  number  of  smaller  personal  computers.  A  brief  history  of  the 
evolution  and  evaluation  of  centralized  and  decentralized  computer 
processing  follows. 


Centralized  Systems 

The  earliest  management  information  systems  were  implemented  on 
large,  centralized  computers  called  mainframes.1  As  a  college  student  in 
the  early  1970s,  I  gained  my  first  exposure  to  these  mainframe  machines. 
The  IBM  370  series  computer  was  controlled  and  operated  in  a  highly 
centralized  manner.  Each  day  computer  science  students  would  line  up  in 
front  of  a  small  window,  clutching  a  packet  of  computer  punch  cards.  When 
we  reached  the  head  of  the  line,  we  would  hand  these  cards  to  the  graduate 
student  who  operated  the  computer.  Upon  our  return  the  next  day.  a 
printed  computer  output  awaited  us.  Usually  this  output  contained  many 
“error  messages" — in  my  case,  there  were  more  errors  than  I  care  to  recall. 

The  system  we  used  was  mainframe-oriented,  centrally  managed,  and 
employed  batch-processing  methods.  The  system  had  several  advantages 
and  disadvantages.  The  main  advantage  was  the  graduate  student  who 
operated  the  computer.  The  students  may  have  joked  about  his  thick 
glasses  and  diminutive  stature,  but  this  person  was  an  acknowledged 
expert  in  operating  the  computer.  The  IBM  360  was  an  extremely  complex, 
expensive,  and  delicate  piece  of  equipment.2  Had  the  university  permitted 
untrained,  unqualified  students  direct  access  to  such  a  machine,  there  is 
little  doubt  that  we  would  have  succeeded  in  running  up  huge  maintenance 
and  repair  bills.  The  tiny  window  where  the  students  submitted  their 
computer  cards  was,  in  effect,  the  only  entrance  into  a  safe,  sterile  environ¬ 
ment  where  the  computer  was  protected  from  mistreatment  or  abuse. 
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The  second  principal  advantage  of  centralized  processing  was  enhanced 
efficiency  of  the  computer.  At  the  time,  computer  memory  and  storage 
space  was  limited  and  expensive.3  For  example,  the  data  bases  that  the 
students  used  in  running  their  programs  were  stored  on  large  spools  of 
computer  tape  because  there  was  insufficient  computer  memory  or  direct 
access  storage  device  (IBM’s  terminology  for  a  disk  drive)  capacity  to  support 
student  input.4  In  the  centralized  environment  the  computer  operator 
saved  all  of  the  students’  computer  cards  and  input  them  to  the  computer 
at  the  same  time.  This  meant  that  the  computer  tape  only  had  to  be 
mounted  onto  the  computer  once  a  day.  thus  saving  considerable  time  and 
effort  for  the  operators.  Similarly,  the  portion  of  the  computer’s  main 
memory  needed  for  the  students’  programs  could  be  loaded  once,  accessed 
in  a  single  10-minute  computer  run,  and  then  freed  up  for  other  applica¬ 
tions.  There  were  other  advantages,  but  they  applied  mainly  to  the  com¬ 
puter  and  the  computer  staff. 

The  disadvantages  of  the  system  accrued  to  the  user.  Although  the 
computer  was  protected  from  student  mistakes,  the  student  was  not 
allowed  to  learn  to  operate  the  computer.  Thus,  sys  i  expertise  rested 
solely  with  the  operator. 

An  obvious  second  disadvantage  was  the  maximization  of  computer 
efficiency  obtained  at  the  user’s  expense .  When  the  little  window  was  closed 
and  locked,  the  students  were  denied  access  to  the  computer,  and  they  had 
to  wait  around  until  the  computer  center  reopened.  Similarly,  the  24-hour 
wait  for  the  return  of  computer  outputs  was  highly  inconvenient  to  the  user. 
Imagine,  if  you  will,  a  student  program  containing  just  one  or  two  minor 
programming  errors.  If  the  machines  were  instantly  accessible,  the  student 
could  have  checked  the  results  and  corrected  the  errors  in  just  a  few 
minutes.  Instead,  the  system  forced  students  to  wait  in  line  to  input  their 
work  and  then  wait  another  24  hours  to  see  the  results.  Thus,  even  the 
most  trivial  programs  might  take  days  or  even  weeks  to  complete. 

The  disadvantage  that  1  considered  to  be  the  greatest,  however,  was  that 
the  system  lacked  redundancy.  In  other  words,  if  the  system  was  in  a  "STOP 
state,"  there  was  nothing  to  do  but  wait  until  someone  repaired  the 
computer  so  that  processing  could  resume.5  For  a  student  facing  a 
deadline,  this  kind  of  delay  was  most  distressing. 

If  these  centralized  systems  were  so  poorly  developed,  why  did  users 
continue  to  acquire  them — especially  given  the  attitude  of  some  early 
developers  that  users  had  to  be  motivated  to  “swim  hard  to  keep  from 
sinking?"6  Part  of  the  answer  is,  of  course,  that  there  was  not  much  of  an 
alternative.  Until  recently,  the  centrally  located  mainframe  was  the  only 
computer  available,  and  only  computer  professionals  could  operate  and 
maintain  them.7 

The  use  of  centralized  systems  in  the  Air  Force  was  similar  to  use  in  the 
college  environment.  A  primary  advantage  of  a  centralized  system  in  the 
Air  Force  was  standardization.  A  centralized  system  could  meet  the  needs 
of  a  large  organization  by  having  a  standard  information  system.  Thus, 
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personnel  transferring  to  a  different  job  or  moving  to  a  new  location  would 
not  have  to  learn  a  new  system.8 

Centrally  developed  and  implemented  systems  also  provided  cheap, 
long-term  data  storage.  Data  from  many  different  offices  or  locations  could 
be  written  onto  inexpensive  spools  of  computer  tape  and  kept  in  a  clean, 
dry,  and  air-conditioned  storage  area.9 

Large  systems  tended  to  have  good  documentation.  Professionals 
prepared  the  design  and  operating  documents  and  delivered  them  to  the 
user  as  part  of  the  system.  Thus,  when  system  problems  or  software  logic 
errors  (bugs)  arose,  the  logic  and  source  code  used  by  the  original  program¬ 
mers  was  available  to  aid  in  solving  the  problem.10  As  a  rule,  centralized 
data  input  was  accurate  because  it  was  performed  by  experienced 
keypunchers  who  were  able  to  use  accuracy-enhancing  techniques  such  as 
double  punching,  check  digits,  and  hash  totals.11 

The  most  important  advantage  of  the  centralized  system,  however,  was 
that  information  was  a  “corporate  resource” — which  meant  that  it  was 
managed  by  a  professional  data  base  expert  and  was  available  to  everyone 
in  the  organization.  This  system  reduced  the  amount  of  effort  needed  to 
input  data  because  it  is  entered  only  one  time,  yet  it  was  available  to  many 
users.  Additionally,  such  a  system  decreased  the  possibility  that  in¬ 
dividuals  would  create  their  own  “pools  of  data"  to  serve  personal  rather 
than  corporate  goals. 12 

Although  this  system  had  many  advantages,  it  was  frustrating  for  the 
user.  Computer  resources  were  so  limited  and  expensive  that  users  had  to 
make  every  consideration  toward  increasing  the  efficient  use.13  However, 
rapid  advances  in  technology  quickly  changed  this  situation.14 


Decentralized  Systems 

By  the  early  1980s  personal  computers  were  appearing  all  over  the  Air 
Force.15  Like  many  others,  I  quickly  discovered  that  this  new  technology 
had  many  advantages.  The  greatest  advantage  was  that,  for  the  first  time, 
the  machine  existed  solely  to  serve  the  user.  The  long  lines  at  the  computer 
room  window  soon  became  a  distant  memory.  The  personal  computer  had 
the  flexibility  to  fit  its  hours  of  operation  to  the  user’s  schedule.  Just  as 
exciting  was  the  quantum  improvement  in  response  time.  Instead  of 
waiting  overnight  for  the  results  from  a  program,  I  could  have  my  answers 
in  a  matter  of  seconds. 

Another  advantage  was  the  amount  of  memory  and  direct-access  storage 
which  rapidly  became  available  on  the  new  desktop  computers.  In  1965 
computer  memory  was  very  expensive,  and  a  large  machine  had  perhaps 
4K  of  memory.16  In  1984  the  Z-100  computer  had  444K  of  memory  plus 
720K  of  floppy  disk  storage.  It  also  had  more  capabilities  and  reliability 
than  the  1964  model  EN1AC,  which  weighed  28  metric  tons  and  was  the 
size  of  a  six-room  house.17  By  1988  this  machine  was  replaced  by  a  Z-248 
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computer  that  had  over  1M  of  memory  plus  40M  of  hard  disk  (five  times 
more  real  storage  than  the  old  IBM  370). 18  This  new  machine  had 
capabilities  similar  to  the  earlier  mainframes,  yet  it  was  inexpensive  enough 
for  each  operator  to  own. 

A  final  advantage  was  cost.  The  new  Z-248  cost  less  than  $4,000.  which 
was  only  a  fraction  of  the  $1  million  that  the  university  had  allegedly  paid 
to  install  the  old  IBM  370  series  mainframe. 1 9  Even  after  adding  accessories 
such  as  a  top-of-the-line  printer,  a  modem,  and  lots  of  advanced  software, 
the  personal  computer  was  a  bargain.  This  low  cost  enabled  organizations 
to  purchase  many  of  the  smaller  computers,  and  it  had  a  built-in  redun¬ 
dancy  that  protected  the  user  from  computer  failure.  If  one  computer 
happened  to  malfunction,  the  user  could  simply  move  the  data  files  to 
another  nearby  computer,  run  the  program,  and  continue  working.  Even 
though  there  were  many  advantages  of  personal  computers,  there  were  also 
some  disadvantages. 

A  primary  disadvantage  was  that  there  was  no  real  expert  around  to  help 
solve  a  computer  problem.20  The  highly  skilled  operator  in  the  mainframe 
computer  room  was  replaced  with  informal  “computer  gurus,"  some  of 
whom  were  knowledgeable,  and  others  who  caused  more  problems  than 
they  solved.21 

Even  the  improved  availability  and  faster  response  time  caused  an 
unexpected  problem — unrealistically  high  user  expectations.  In  the  past 
users  had  been  content  to  wait  24  hours  for  a  computer  product,  but  by 
1988  they  would  not  tolerate  any  delay  of  more  than  a  few  seconds.  In  fact, 
when  the  new  Z-248  computers  arrived  in  1988,  one  of  their  main  selling 
points  was  that  they  were  much  faster  than  the  supposedly  obsolete  Z- 1 00s. 

Sole  ownership  of  the  computer  also  had  its  disadvantages.  Although 
the  user  controlled  the  machine,  there  was  no  one  to  safeguard  against 
mistakes.  Thus,  a  user  who  forgot  to  make  backup  copies  of  a  large  program 
ran  the  risk  of  having  to  recreate  the  program  from  scratch.  In  the  old 
mainframe  days  the  operators  took  care  of  the  backup  requirements.22 

Perhaps  the  greatest  drawback  of  personal  computers  is  the  tendency  for 
users  to  automate  tasks  that  they  should  perform  manually.23  In  my  case, 
this  “data  deluge"  is  best  exemplified  by  a  program  I  wrote  to  identify  the 
highest  priced  item  from  a  list  of  30  items.  4  The  program  ran  flawlessly, 
and  I  was  quite  proud  of  myself  until  a  wiser  coworker  pointed  out  that  a 
six-year-old  child  could  do  the  same  job  manually  in  less  time. 

When  personal  computers  arrived  on  the  scene,  a  new  kind  of  manage¬ 
ment  information  system  (MIS)  appeared.  The  American  National  Stan¬ 
dards  Institute  (ANSI)  defined  management  information  systems  as 
“management  performed  with  the  aid  of  automatic  data  processing."25 
Computer  users  quickly  discovered  that  with  the  aid  of  application  software 
packages  such  as  Condor.  DBase,  or  Enable,  they  could  design  and  use 
highly  personalized  MISs.  The  advantages  of  such  “user-driven  computing" 
were  obvious.26 
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Personal  computers  were  highly  responsive  to  the  user.  With  a  personal 
computer,  you  had  instant  access  to  the  data  in  the  system  and  could  sort, 
select,  or  arrange  it  in  literally  the  blink  of  an  eye.27  Thus,  it  is  easy  to  see 
why  people  would  prefer  this  rapid  response  to  the  old  method  of  daily, 
weekly,  or  monthly  reports. 

Personal  computers  also  greatly  enhanced  flexibility.  A  user  could 
customize  computer  output  to  meet  specific  requirements.  Printed  reports 
could  be  tailored  to  include  the  needed  information  for  a  given  decision 
without  any  extraneous  data. 

Unfortunately,  user  enthusiasm  for  the  system  overshadowed  many 
negative  factors  (disadvantages  and  risks).28  In  my  opinion,  the  greatest 
disadvantage  was  that  information  began  to  be  seen  as  a  personal  rather 
than  a  corporate  asset.  This  philosophy  directly  conflicted  with  the  tradi¬ 
tional  view  of  the  use  of  information  in  an  organization.29  However,  there 
were  other  drawbacks  that  were  not  so  obvious. 

Duplication  of  effort  became  a  real  problem.  Personalized  information 
systems  required  personalized  data  entry.  Thus,  it  became  possible  for  the 
same  piece  of  data  to  be  entered  individually  into  numerous  systems  on 
many  different  machines.  Managers  began  to  express  concern  that 
employees  were  spending  too  much  time  as  data-entry  clerks  and  not 
enough  time  doing  their  Jobs.  Moreover,  many  personalized  MISs  were  just 
“one-shot"  applications  in  which  a  single  report  did  not  justify  its  program¬ 
ming  effort. 

Programming  errors  also  became  more  prevalent  as  programming  was 
performed  by  well-meaning  amateurs  instead  of  professional  programmers. 
Indeed,  “powerful  computers"  were  "capable  of  powerful  mistakes."31  An 
excellent  example  of  this  type  of  risk  is  the  celebrated  legal  case  of  James 
A.  Cummings.  Inc.  versus  Lotus  Development  Corporation,  where  the 
plaintiff  unsuccessfully  asserted  that  an  error  in  program  logic  had  caused 
his  construction  company  to  underbid  a  proposal  by  $254,000.  The  court 
found  that  the  expensive  error  was  the  fault  of  a  Cummings  employee  who 
had  developed  a  faulty  worksheet.32 

Documentation  and  continuity  were  other  problem  areas.  Most  of  the 
designers  of  these  personalized  systems  were  not  computer  professionals, 
and  they  did  not  understand  or  appreciate  the  need  for  proper  documenta¬ 
tion.33  This  created  a  continuity  problem  because  only  the  individual  who 
created  the  program  could  use  it  or  make  corrections  to  it.  However,  when 
that  person  left  the  organization,  the  MIS  that  had  enhanced  productivity 
suddenly  became  an  unmaintainable  liability. 

New  security  problems  also  arose.  Whereas  the  centralized  computer 
facility  was  relatively  safe  from  unauthorized  Intruders,  personal  computers 
were  located  in  open  areas  or  in  individual  offices.  As  a  result,  the  MIS  was 
more  vulnerable  to  tampering,  theft,  and  unauthorized  access.  Worse  still, 
the  growing  practice  of  swapping  software  among  personal  computers  made 
them  vulnerable  to  “viruses."34 
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Computer  viruses  are  not  a  new  phenomenon,  nor  are  they  a  problem 
.  that  is  unique  to  personal  computers.  An  example  of  this  virus  was  the 
Multiplexed  Information  and  Computing  Service  (MULTICS)  system,  an 
early  US  Air  Force  MIS  which  was  touted  as  being  invulnerable  to  un¬ 
authorized  access.  In  a  test  “raid."  Air  Force  Lt  Col  Roger  R.  Schell  and 
Steven  B.  Lipner  used  a  program  to  defeat  the  security  of  MULTICS.  This 
program  was  a  virus  that  tricked  the  host  computer  into  duplicating 
classified  information  and  routing  the  copy  to  unprotected  areas  of  com¬ 
puter  memory.35  Personal  computers  are,  however,  more  vulnerable  to 
virus  attacks  because  the  virus  can  be  introduced  either  through  shared 
computer  diskettes  or  through  the  ever-growing  networks  that  tie  many 
computers  together. 

As  the  cost  of  computer  hardware  has  plummeted,  the  availability  and 
capabilities  of  computers  have  skyrocketed.36  This  revolution  in  computing 
has  directly  impacted  the  way  management  information  systems  are 
designed. 


Early  Design  Approach 

Management  information  systems  are  not  new,  and  automated  systems 
have  been  with  us  for  some  time.37  These  earlier  systems  used  large, 
centralized  mainframe  computers  forcing  reliance  on  centralization  that 
influenced  the  system  design  process. 

Although  there  was  never  a  standard  process  for  the  design  and  use  of 
management  information  systems,  the  step-by-step  procedures  outlined  by 
most  experts  were  remarkably  similar.  Basically,  the  design  approach  was 
as  follows: 

1 .  The  user  organization  identified  the  types  of  information  required  to 
do  its  job. 

2.  A  systems  analyst,  an  expert  in  MIS  design  and  implementation, 
assessed  the  situation. 

3.  The  analyst  discussed  the  needs  with  the  user. 

4.  The  analyst  performed  a  “feasibility  study"  and  formulated  a  proposed 
system  to  meet  the  user's  needs. 

5.  Once  the  user  accepted  the  proposal,  system  development  began. 

6.  The  developer  tested  the  new  system  and  delivered  it  to  the  user.38 

The  resultant  systems  generally  worked  in  a  technical  sense,  but  they 
did  not  always  meet  the  expectations  of  the  user.  As  many  as  75  percent 
of  all  large  systems  were  described  as  “operating  failures."39  One  United 
Research  Company  poll  of  92  chief  e:  outive  officers  found  that  70  (76 
percent)  either  “agreed"  or  “strongly  agreed"  that  “decisions  regarding 
resource  allocation  for  information  systems  are  typically  based  on  promises 
of  benefits  or  service  that  are  seldom  achieved."40  In  other  words,  the 
system  worked  fine,  it  Just  did  not  meet  the  user’s  needs  or  expectations. 
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The  problem  was  not  that  analysts  were  incompetent,  they  simply  were 
technicians  and  were  not  well  acquainted  with  the  business  of  the  users.41 
As  one  expert  stated: 

The  difficulty  with  the  traditional  approach  is  the  lack  of  participation  by.  and 
interaction  with,  the  end  user.  The  difficulties  are  exemplified  by  the  classic  series  of 
drawings  that  depict  the  design  versus  the  implementation  of  a  child's  swing  in  a 
tree — you  know,  the  one  that  shows  a  precisely  illustrated,  level  swing  in  the  first  panel 
and  an  obviously  askew  board  hanging  from  two  uneven  ropes  in  the  second.4* 

Many  problems  could  be  cited  to  Illustrate  the  frustrations  of  these 
dissatisfied  users.  One  significant  problem  was  that  the  “standard  reports," 
which  many  automated  systems  produced,  were  difficult  to  adapt  to  the 
special  requirements  or  desires  of  individual  users.43  Often  these  reports 
were  bulky  and  contained  too  much  detailed  information.  Typically  they 
contained  a  number  of  ciyptic  abbreviations.  These  abbreviations  reduced 
the  workload  on  the  central  computer,  but  they  also  required  extensive 
training  for  the  user.  This  “prespecified  computing"  proved  “slow  and 
expensive  in  person-power."44 

Timeliness  was  also  a  problem.  Often  there  was  only  one  printer  in  the 
computer  room,  and  reports  had  to  be  printed,  collated,  and  manually 
distributed  to  the  users.  Thus,  it  could  take  several  days  for  data  to  reach 
the  desk  of  the  person  who  needed  it.45  Another  important  issue  was  that 
centralized  control  of  the  management  information  system  put  the  organiza¬ 
tion  in  a  vulnerable  position  if  any  harm  befell  the  computer.  By  placing 
all  of  their  informational  eggs  in  a  single  basket,  an  organization  that  relied 
heavily  on  its  MIS  risked  catastrophe.  Therefore,  maintaining  backup  data 
and  data-processing  capabilities  became  necessary.46 

Finally,  security  for  centralized  systems  proved  to  be  a  mixed  blessing. 
Physical  security  was  enhanced  by  having  all  of  an  organization’s  data  and 
hardware  in  a  single,  easily  secured  area.47  Electronically,  however, 
centralized  data  was  vulnerable.  Perhaps  (he  most  famous  example  of  this 
vulnerability  was  the  MULT1CS  system.  Schell  and  Lipner  penetrated  the 
system  and  siphoned  off  classified  information  faster  than  the  raiding 
computer  could  process  it.  The  perpetrators  actually  had  to  program  their 
computer  to  steal  only  the  “best"  information  and  ignore  less  important 
data.48 

Thus,  it  can  be  argued  that  the  evolution  of  computers  has  produced  two 
types  of  MIS  (the  centralized  mainframe  system  and  the  decentralized 
personal  computer),  and  that  each  type  has  its  own  advantages  and 
disadvantages.  Unfortunately,  new  system  design  criteria  that  combine  the 
best  methodologies  of  both  systems  are  just  beginning  to  be  proposed. 


New  Design  Approach 

Early  efforts  to  apply  the  centralized  mainframe  system  development 
methodologies  to  personal  computers  produced  less  than  optimal  results. 
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In  1985  I  reviewed  a  major  system  development  program  that  tried  to 
combine  an  older  centralized  system  with  state-of-the-art  personal  com¬ 
puters.  One  problem  was  that  the  centralized  system  was  being  used  to 
turn  on  the  personal  computers  at  eight  o’clock  each  Monday  and  have 
them  print  a  large,  standardized  report  which  In  no  way  used  the  capability 
of  the  new  machines  to  customize  the  data  output.  Another  problem  with 
the  new  system  was  that  the  data  input  was  limited  to  80  characters  per 
line.  This  constraint  required  the  users  to  look  up  and  input  numerous 
coded  entries.  This  limitation  was  unnecessary.  It  was  simply  a  carryover 
feature  from  the  days  of  the  80-column  punch  card.49 

These  design  problems  were  not  limited  to  large,  mainframe-oriented  MIS 
developments.  For  example,  I  will  relate  my  experience  in  developing  a 
small,  single-office  system  for  reporting  audit  time  expenditures.  In 
developing  this  system  I  used  a  Zenith  120  series  personal  computer  to 
assist  the  secretarial  staff  in  submitting  monthly  reports  to  AFAA  head¬ 
quarters.  Unfortunately,  in  my  zeal  to  develop  a  perfect  system,  I  committed 
several  serious  design  errors. 

My  worst  mistake  was  falling  to  document  the  logic  and  design  of  the 
system,  like  many  novices.  I  was  so  preoccupied  with  completing  and 
delivering  the  system  that  I  neglected  documentation.50  I  also  failed  to  plan 
for  the  optimal  efficiency  of  the  system  by  forgetting  that  “simple  is  the  sign 
of  skill.’51  The  software  package  had  a  number  of  interesting  command 
options,  and  I  was  determined  to  use  all  of  them.  Thus  I  delivered  a  system 
that  contained  a  number  of  unnecessary  sorts,  merges,  tabulates,  and  so 
on. 

Finally,  I  never  got  around  to  making  adequate  backup  copies  of  the 
system.  As  was  the  case  with  documentation,  making  a  backup  copy  was 
a  low-prlorlty  task  that  could  wait  until  later.  Once  again,  later  never 
seemed  to  arrive.52 

The  results  were  catastrophic.  The  poorly  designed  software  logic  took 
too  long  to  produce  the  required  reports.  Naturally,  the  computer  did  not 
complain  about  performing  its  functions  for  30  minutes,  but  the  secretaries 
were  unhappy.  Without  proper  documentation,  I  became  the  source  for 
answering  questions  about  how  to  operate  the  system.  Worst  of  all,  the 
lack  of  backup  copies  did  not  become  a  problem  until  the  disks  were 
accidentally  erased,  and  by  then  it  was  too  late  to  correct  the  situation. 

The  problems  previously  described,  plus  a  host  of  others,  are  attributable 
to  two  factors:  ’the  undereducated  manager  who  lives  only  in  the  world  of 
business  and  the  reclusive  MIS  developer  who  speaks  primarily  com¬ 
puterese.’53 

Clearly,  organizations  need  a  system  design  methodology  that  can  com¬ 
bine  the  best  features  of  large,  centrally  designe  d  systems  with  those  of 
personal  computers.  In  fact,  the  AFAA  should  explore  the  use  of  this  design 
approach  because  it  possesses  both  a  modem,  mainframe  system  and  a 
large  number  of  personal  computers.  Hopefully,  we  can  avoid  many  system 
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design  problems  that  stem  from  “traditional  MIS  departments  and  an 
explosively  growing  end  user  community  ...  at  perpetual  loggerheads."54 


Air  Force  Audit  Agency  Initial  Design  Steps 

Based  on  this  new  design  approach,  the  AFAA  can  develop  an  effective 
management  information  system  if  it  follows  these  initial  steps: 

1.  Determine  what  Information  AFAA  users  need  to  do  their  jobs 
thoroughly  and  efficiently.  Given  the  mission  of  the  AFAA  and  the  variance 
of  Job  skills  and  requirements  discussed  in  chapter  3,  this  analysis  will  be 
time-consuming.  Nonetheless,  the  analysis  must  be  completed  before  a 
logical  methodology  can  be  instituted. 

2.  Determine  how  well  these  information  needs  are  currently  being  met. 
A  system  designer  should  avoid  fixing  things  that  are  not  broken. 

3.  Establish  criteria  for  determining  which  data  needs  can  be  addressed 
by  the  AFAA’s  mainframe  computer  system  and  those  which  can  be  met  by 
the  distributed  capabilities  of  personal  computers. 

4.  Designate  an  AFAA  to  manage  MIS  development.  This  agency  would 
ensure  that  information  requirements  are  met,  limit  duplication  of  data 
within  the  agency,  and  monitor  system  interaction. 

Chapter  3  describes  the  agency’s  organization  and  identifies  system 
factors  that  affect  the  design  process.  This  information  will  guide  future 
system  development. 
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Chapter  3 


Organizational  Factors 


If  all  organizations  used  the  same  information  in  the  same  way,  identify¬ 
ing  management  information  system  (MIS)  requirements  would  be  easy.  In 
fact,  some  software  house  could  write  a  “generic  MIS"  which  any  organiza¬ 
tion  could  buy  off  the  shelf  to  satisfy  its  information  needs.  Unfortunately, 
each  organization  is  unique  and  has  different  information  requirements. 
Therefore,  before  identifying  the  information  requirements  of  an  organiza¬ 
tion.  you  must  understand  its  mission,  structure,  and  impact  on  Air  Force 
operations.  Also,  one  must  understand  the  requirements  of  the  end  users 
of  the  system.  Accordingly,  this  chapter  describes  the  Air  Force  Audit 
Agency’s  mission,  organization,  auditor  qualifications  and  experience,  im¬ 
pact  of  its  reports  on  the  rest  of  the  Air  Force,  and  factors  that  affect  system 
requirements. 


Mission 


The  AFAA  has  the  distinction  of  being  chartered  under  public  law  as  an 
original  and  permanent  component  of  the  Air  Force.  The  National  Security 
Act  of  1947  charges  the  secretary  of  the  Air  Force  (SAF)  to  “cause  internal 
audit  to  be  organized  and  conducted."1  The  AFAA's  mission  is  to  "evaluate 
the  effectiveness  and  efficiency  of  Air  Force  management  at  all  levels  of 
command."2  With  the  exception  of  certain  Army  and  Air  Force  Exchange 
Service  audits,  which  are  performed  under  AFAA  supervision,  the  AFAA  is 
the  only  organization  authorized  to  perform  audit  functions  for  the  Air 
Force.3  Originally,  it  was  established  under  the  jurisdiction  of  the  comp¬ 
troller  general  of  the  Air  Force;  however,  because  of  the  sensitive  nature  of 
the  audit  mission  and  an  implicit  need  for  managerial  independence,  it  was 
later  reorganized  as  a  separate  operating  agency  (SOA).  To  meet  American 
Institute  of  Certified  Public  Accountants  (AICPA)  and  General  Accounting 
Office  (GAO)  standards  of  independence,  the  auditor  general  of  the  Air  Force 
now  reports  directly  to  the  secretary  of  the  Air  Force  and  receives  policy 
direction  from  the  DOD  Inspector  general  (DODIG). 
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Organization 


Because  military  officers  were  perceived  as  being  vulnerable  to  the 
influence  of  any  senior-ranking  officer,  the  Job  of  auditor  general  of  the  Air 
Force  was  converted  to  a  civilian  position.  The  current  auditor  general  is 
John  Boddie,  a  senior  executive  service  (SES)  employee  with  headquarters 
at  Norton  AFB,  California.4 

Military  command  of  the  agency  resides  with  the  deputy  auditor  general 
of  the  Air  Force,  Brig  Gen  Basil  Pflumm.  General  Pflumm  is  assigned  to  the 
Pentagon  and  maintains  audit  liaison  with  the  Air  Staff.5 

Because  the  Air  Force  is  a  highly  diversified,  worldwide  organization,  the 
AFAA  has  adopted  an  organizational  structure  that  serves  the  Air  Force  at 
headquarters  and  operational  levels  (fig.  1).  Consequently,  AFAA  has  two 
headquarters  directorates  and  three  operational  directorates.  Two  of  the 
operational  directorates  provide  audit  service  to  major  commands  and 
Headquarters  USAF;  one  audits  base-level  activities. 

Headquarters  Directorates 

The  Directorate  of  Resource  Management  (AFAA/RM) — located  at  AFAA 
headquarters,  Norton  AFB.  California — provides  centralized  control  of  the 
AFAA  budget,  supply  activities,  and  administrative  services.  This  direc¬ 
torate  also  performs  management  functions  for  both  military  and  civilian 
personnel. 

The  Directorate  of  Operations  (AFAA/DO),  also  located  at  Norton  AFB, 
consists  of  three  divisions  that  monitor  AFAA  operations  at  all  levels.  The 
Operations  Division  (AFAA/DOO)  provides  overall  audit  planning  guidance 
that  ensures  high-quality  audit  projects.  This  division  coordinates  audit 
projects  with  other  government  agencies  such  as  the  GAO  or  DODIG  to 
make  certain  that  audit  coverage  is  not  wasted  by  duplicating  recent  or 
ongoing  efforts.  It  also  monitors  the  resolution  of  major  issues  such  as 
fraud,  violations  of  the  law,  management  nonconcurrence  with  audit  find¬ 
ings,  or  large  potential  monetary  benefits  that  arise  from  AFAA  audits. 

The  Technical  Services  Division  (AFAA/ DOT)  has  highly  skilled 
specialists  in  various  areas.  These  experts  ensure  that  AFAA  audits  are 
“conducted  by  staff  who  collectively  have  the  knowledge  and  skills  neces¬ 
sary  to  conduct  the  audit."6  This  division  employs  computer  scientists, 
engineers,  contracting  experts,  a  statistician,  and  an  operations  researcher. 
In  the  event  that  an  audit  requires  technical  expertise  in  areas  not  repre¬ 
sented  on  the  DOT  staff,  that  is,  law  or  medicine,  this  division  can  contract 
for  additional  support. 

The  Standardization  and  Internal  Affairs  Division  (AFAA/DOV)  audits  the 
auditors.  This  division  conducts  operational  reviews  to  ensure  that  audits 
are  conducted  under  established  policies  and  procedures. 
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Operational  Directorates 

The  operational  audit  mission  is  primarily  assigned  to  three  directorates: 
Financial  and  Support  Directorate,  Acquisition  and  Logistics  Directorate, 
and  the  Field  Activities  Directorate.  They  are  responsible  for  the  perfor¬ 
mance  of  audits  and  the  reporting  of  audit  results  to  the  appropriate 
command  level. 

The  Financial  and  Support  Directorate  (AFAA/FS)  is  located  at  Norton 
AFB  and  maintains  subordinate  field  offices  at  six  major  Air  Force  bases. 
It  is  organized  along  functional  lines  and  performs  Air  Force-wide  audits  in 
such  diverse  areas  as  standard  computer  systems:  communications:  per¬ 
sonnel;  morale,  welfare,  and  recreation;  support  services;  and  major  train¬ 
ing  missions.  This  directorate  performs  multilocation  audits  and  reports 
its  findings  to  the  secretary  of  the  Air  Force  via  summary  reports  of  audit 
(SRAs).  Auditors  assigned  to  this  directorate  are  selected  based  on 
demonstrated  performance  within  the  agency  or  other  government  audit 
organizations. 

The  Acquisition  and  Logistics  Directorate  (AFAA/QL)  is  headquartered  at 
Wright -Patterson  AFB.  Ohio.  Like  the  Financial  and  Support  Directorate. 
AFAA/QL  performs  multilocation.  Air  Force-^wide  audits  and  issues  reports 
to  the  SAF.  It.  however,  specializes  in  the  multibillion-dollar  business  of 
logistics.  Thus  these  auditors  have  extensive  audit  experience  in  the 
logistics  environment.  This  directorate  maintains  subordinate  field  offices 
at  14  air  logistics  centers  and  Air  Force  Systems  Command  bases. 

The  Field  Activities  Directorate  (AFAA/FD)  is  situated  at  Norton  AFB,  but 
its  field  offices  are  geographically  dispersed  to  Air  Force  bases  worldwide. 
These  offices  are  tenant  activities  at  46  major  bases  and  provide  audit 
services  to  all  other  bases  on  a  TDY  basis.  Additionally,  senior-ranking 
auditors,  assisted  by  relatively  large  audit  stalls,  are  located  at  each 
MAJCOM  and  provide  special  audit  services.  AFAA/FD  offices  perform 
their  audits  at  a  single  location  and  report  their  results  to  the  local 
commander  and  the  appropriate  MAJCOM  commander.  Journeyman 
auditors  assigned  to  these  bases  form  the  backbone  of  the  agency,  and  they 
compete  for  promotion  to  supervisory  or  audit  management  positions. 


Auditor  Qualifications  and  Experience 

AFAA  auditors  are  assigned  duties  at  all  levels  of  management .  and  even 
the  highest-ranking  audit  executives  began  their  careers  as  base  level 
auditors.  Approximately  75  percent  of  all  AFAA  auditors  are  civilians  whose 
grades  range  from  GS-5  auditor  trainees  to  executives.  All  newly  hired 
auditors  must  be  graduates  of  a  four-year  college  or  university  and  must 
have  a  minimum  of  24  semester  hours  in  accounting.  Military  officers  in 
the  grades  of  second  lieutenant  to  brigadier  general  make  up  the  remainder 
of  the  auditor  corps.  Some  of  these  officers  spend  their  entire  careers 
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assigned  to  AFAA  while  others  have  cross-trained  from  other  specialties. 
Over  40  percent  of  all  auditors  hold  a  master’s  degree. 

Whether  they  are  military  or  civilian,  however,  all  AFAA  auditors  must 
complete  80  hours  of  continuing  professional  education  every  two  years. 
For  many  this  continuing  education  results  in  professional  certification  as 
a  certified  public  accountant  (CPA),  certified  internal  auditor  (CLA),  or 
certified  information  systems  auditor  (CISA).  Nearly  30  percent  are  CPAs. 
CIAs,  or  CISAs. 

Of  importance  to  this  report  is  the  unusually  high  level  of  computer 
literacy  among  auditors.  All  auditors  are  classified  according  to  their 
knowledge  of  computers  and  automated  systems  and  receive  extensive 
specialized  training  in  accessing,  operating,  and  auditing  automated  sys¬ 
tems.  The  minimum  level  of  computer  competence  required  of  AFAA 
auditors  is  certification  as  a  “skill  level  1"  automatic  data  processing  (ADP) 
auditor.  Level  1  auditors  possess  an  “overall  awareness  of  the  electronic 
data  processing  (EDP)  environment,  personal  computers.  Air  Force  sys¬ 
tems.  and  areas  of  vulnerability.”7  They  have  received  specialized  training 
at  in-house  audit  schools,  have  graduated  from  the  ATC  Auditor  Systems 
Retrieval  Course  at  Sheppard  AFB,  Texas,  and  have  completed  ATC- 
developed  courses  in  personal  computers  and  software. 

Certification  as  a  level  2  auditor  presupposes  knowledge  at  skill  level  1 
and  requires  the  ability  to  “participate  in  system  development  audits, 
evaluate  internal  controls  in  major  systems,  discover  potential  areas  of 
fraud,  waste  and  abuse,  and  adapt  and  use  software  programs  to  test  the 
accuracy  of  EDP  systems  being  audited."8  Level  2  auditors  attend  a  course 
in  residence  at  California  Polytechnic  Institute.  Currently,  over  25  per cent 
of  the  agency’s  auditors  are  certified  at  skill  level  2. 

Skill  level  3  auditors  specialize  in  computer  auditing  and  have  “specific 
expertise  in  operating  systems,  data  base  management  systems,  software 
security,  hardware /software  controls,  data  communication  and  network¬ 
ing."9  They  are  graduates  of  a  contracted  in-house  school  specializing  in 
data  communications  and  networking  and  many  have  received  additional 
training  at  local  universities,  the  DOD  Computer  Institute,  or  selected  ATC 
courses.  Approximately  10  percent  of  AFAA  auditors  have  been  trained  and 
certified  to  skill  level  3. 


Impact  of  Mission  on  Air  Force  Operations 

Although  many  people  think  of  the  AFAA  as  an  organization  with  people 
that  count  cash  and  balance  financial  statements,  its  role  is  actually  much 
greater  than  that.  AFAA  auditors  “independently  and  objectively  analyze, 
review  and  evaluate  existing  procedures  and  controls  relating  to  all  Air  Force 
organizations,  activities,  programs,  and  functions.”10  Although  the 
auditors  do  not  function  as  area  experts — who  tell  operational  personnel 
how  to  do  their  jobs — they  are  experts  at  ensuring  that  government  resour- 
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ces  and  assets  are  used  effectively  and  efficiently.  During  wartime,  auditors 
in  the  combat  theater  assess  the  “adequacy  and  effectiveness  of  support 
furnished  to  combat  forces  and  controls  used  to  prevent  unauthorized 
diversion  of  equipment,  supplies,  and  other  forces.1 

The  agency’s  greatest  Impact  on  Air  Force  operations  is  through  Its 
various  audit  reports  to  commanders  and  managers.  In  fiscal  year  1988, 
AFAA  Issued  1.420  installation-level  reports  to  wing  commanders  that 
suggested  Improvements  in  areas  as  diverse  as  flying-hour  utilization,  spare 
parts  Inventories,  commissary  warehousing,  and  hospital  operations.  An 
additional  1 20  “Commander’s  Audit  Program"  reports  were  Issued  confiden¬ 
tially  to  base  and  wing  commanders  in  response  to  requests  for  audit 
assistance.  At  the  Air  Staff  level,  the  agency  issued  87  reports  to  the  SAF 
on  a  variety  of  operational,  logistics,  and  comptroller  topics. 

In  addition  to  identifying  ineffective  and  inefficient  practices,  AFAA  audit 
reports  often  show  management  ways  to  achieve  mission  objectives  at  a 
lower  cost.  The  savings,  referred  to  as  potential  monetary  benefits  (PMB). 
allow  managers  to  reallocate  scarce  funds  into  other  critical  mission  areas. 
During  fiscal  year  1988,  AFAA  auditors  identified,  and  management  con¬ 
curred  with,  over  $339  million  in  PMBs.  This  averaged  nearly  $412,000 
saved  for  each  of  the  agency’s  823  assigned  auditors,  thus  returning  $10 
in  benefits  to  the  Air  Force  for  each  dollar  of  salary  paid  to  auditors. 


Factors  Affecting  Information  System  Requirements 

Many  factors  affect  AFAA  information  system  requirements.  Unlike  many 
other  small  organizations,  it  has  offices  and  locations  worldwide.  This 
global  network  requires  the  agency  to  c  .Uect,  process,  and  report  Informa¬ 
tion  from  any  Air  Force  location. 

The  diversity  of  the  AFAA  mission  also  complicates  MIS  requirements. 
Like  any  MAJCOM  or  SOA,  AFAA  must  maintain  and  process  information 
relative  to  personnel  management,  budget  issues,  administration,  etc.  In 
addition,  it  requires  and  examines  information  about  virtually  every  or¬ 
ganization.  Thus,  Information  requirements  can  vaiy  tremendously  among 
individual  auditors  and/or  audit  offices. 

AFAA’s  unique  organizational  placement — directly  under  the  SAF  with 
review  and  oversight  authority  by  the  DOD  inspector  general — also  affects 
information  system  requirements,  ft  generates  requirements  to  collect  and 
disburse  information  to  non-Alr  Force  agencies,  which  a  more  traditionally 
placed  small  organization  might  not  have  to  do. 

Ironically,  AFAA’s  success  at  identifying,  reporting,  and  correcting 
deficiencies  and  at  discovering  more  cost-effective  ways  to  achieve  the  Aft- 
Force  mission  further  complicates  MIS  requirements.  In  fact,  $339  million 
in  cost  benefits  generates  a  great  deal  of  interest  from  congressional  staffers, 
civilian  news  organizations,  and  DOD  agencies.  As  a  result,  numerous 
requests  flow  in  for  audit  reports,  audit  resource  expenditures,  audit 
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methodologies,  etc.  Thus,  the  agency  must  frequently  maintain,  compile, 
and  report  information  merely  to  satisfy  the  needs  of  non-AFAA  organiza¬ 
tions. 

Finally,  the  high  level  of  computer  literacy  among  AFAA  personnel  also 
affects  MIS  requirements.  AFAA  auditors  understand  automation;  there¬ 
fore,  they  expect  many  operational  features  from  their  automated  systems. 
Chapter  4  will  discuss  these  expectations  in  relation  to  the  Incredible 
computing  power  available  within  the  agency. 
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Chapter  4 


Hardware  and  Software 


Chapter  3  described  the  complex,  worldwide  mission  of  the  agency,  the 
computer  expertise  of  assigned  personnel,  and  the  impact  of  these  factors 
on  information  requirements.  There  is  one  other  factor  that  impacts  on  an 
organization's  information  needs;  namely,  the  automated  tools  which  are 
available  to  retrieve,  store,  and  process  information.  This  chapter  discusses 
the  suite  of  computer  hardware  and  software  available  to  agency  personnel 
to  perform  their  audit  mission  and  examines  some  of  the  common  ways 
auditors  use  these  devices  to  increase  productivity. 


Hardware 

AFAA  is  a  small  organization,  but  in  terms  of  automation  it  is  something 
of  a  giant.  Auditors  have  found  that  the  computer  acts  as  a  “force 
multiplier”  of  audit  effectiveness,  thus  they  have  rushed  to  acquire  and  use 
these  automated  systems.  Because  of  the  unique  needs  of  the  agency,  it 
extensively  uses  both  mainframe  and  personal  computer  systems. 

Mainframe  Systems 

As  of  1989,  the  backbone  of  the  agency’s  hardware  inventory  consists  of 
two  Wang  versus  mainframe  computers,  one  located  at  AFAA  headquarters 
(Norton  AFB,  California)  and  the  other  at  the  Acquisition  and  Logistics 
Directorate  (Wright-Patterson  AFB,  Ohio).  These  large  computers  serve  a 
number  of  critical  functions.  They  provide  Inexpensive,  reliable,  and  long¬ 
term  storage  for  agency  data  because  of  their  large  capacity  tape  and  disk 
drives.  Additionally,  they  provide  larger  computing  power  that  allows  timely 
production  of  standardized  monthly  reports.  Because  the  mainframes  are 
located  within  the  directorates  that  perform  multisite,  Air  Force-wide  audits 
(AFAA/FS  and  AFAA/QL),  they  also  perform  a  valuable  word  processing 
and  data  communication  function.  Obviously,  because  the  SAF  receives 
these  audit  reports  and  because  they  impact  on  operations  throughout  the 
Air  Force,  they  undergo  a  considerable  amount  of  scrutiny  and  review  before 
they  are  issued.  For  the  sake  of  efficiency,  the  agency  has  connected  the 
two  mainframe  computers  so  that  reports  can  be  teleprocessed  between 
bases.  Thus,  these  computers  allow  each  Air  Force-level  report  of  audit  to 
be  instantly  circulated  between  the  performing  directorate  and  the  head- 
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quarters.  The  Wang  computers  also  support  interoffice  word  processing  for 
Headquarters  AFAA,  AFAA/FS,  and  AFAA/QL. 

Although  the  two  Wang  computers  perform  well  in  supporting  the 
agency's  centralized  directorates  (HQ  AFAA.  AFAA/FS.  and  AFAA/QL),  the 
cost  of  using  them  to  provide  similar  capabilities  to  the  field  offices  would 
be  prohibitive.  This  is  one  reason  that  personal  computers  have  become 
increasingly  important  to  the  AFAA  mission. 

Personal  Computers 

The  agency  u^cs  personal  computers  both  in  support  of  the  Wang 
mainframes  and  as  stand-alone  units  located  throughout  the  world  (appen¬ 
dix  A).  To  support  less  than  900  assigned  personnel,  the  agency  has 
obtained  over  500  personal  computers.  Even  the  smallest  field  office  may 
have  six  or  more  personal  computers  on  hand.  The  predominate  type  of 
computer  in  use  is  the  Zenith  (Z). 

As  part  of  the  original  Air  Force-standard  personal  computer  purchase, 
the  agency  obtained  Z-120  personal  computers.  Although  auditors  con¬ 
sidered  the  capabilities  of  these  computers  state  of  the  art  in  1983,  they 
lack  the  memory  and  storage  capacity  needed  for  larger  data  manipulations. 
Currently,  auditors  primarily  use  these  machines  for  word  processing  of 
small  reports  and  manipulation  of  small  audit  spreadsheets. 

Zenith-248  personal  computers  replaced  the  Z-  120s  as  the  “computer  of 
choice"  at  most  agency  locations.  Because  the  Z-248s  are  considerably 
faster  and  more  powerful  than  the  Z-120s,  they  have  taken  over  the  larger 
word  processing,  data  manipulation,  and  telecommunications  functions  at 
base-level  audit  offices.  Audit  managers  at  AFAA/FS  and  AFAA/QL  also 
use  these  computers  extensively. 

Auditors  have  found  many  innovative  uses  for  the  Z-248.  Often  they  use 
the  Z-248  for  downloading  data  from  large  mainframe  computers.  Once 
auditors  download  the  data  to  the  Z-248,  they  can  use  various  data  analysis 
programs  to  look  for  trends  or  abnormalities  within  the  system  being 
audited.  Armed  with  this  analysis,  auditors  can  concentrate  their  efforts 
on  areas  with  the  greatest  audit  potential.  The  Field  Activities  Directorate 
(AFAA/FD)  uses  the  Z-248  to  collect  audit  resource  expenditure  data, 
consolidate  it.  and  send  it  to  both  Headquarters  AFAA  and  AFAA/FD 
regional  offices.  This  enables  senior  supervisory  and  executive  personnel 
to  monitor  the  audit  productivity  of  each  office  and  auditor,  and  it  also 
allows  each  chief  to  remain  abreast  of  office  performance.  Finally,  auditors 
use  the  Z-248s  for  typing  base-level  audit  reports  and  for  transmitting  them 
to  and  from  regional  headquarters.  This  gives  regional  supervisors  the 
ability  to  review,  edit,  and  approve  draft  audit  reports  before  they  are 
presented  to  the  local  commander.  This  ensures  a  high-quality,  timely 
audit  product. 

The  Z-184,  a  portable,  laptop  computer,  has  most  of  the  capabilities  of 
the  Z-248.  The  Z-184  is  popular  among  audit  managers  at  AFAA/FS  and 
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AFAA/QL  because  they  can  take  their  work  with  them  as  they  travel  to  job 
sites,  audit  conferences,  or  the  Pentagon.  Also,  its  built-in  teleprocessing 
capability  allows  the  auditor  to  send  or  receive  time-sensitive  information 
while  on  the  road. 


Software 

The  AFAA  has  standardized  “application  software"  packages  so  that 
auditors  can  effectively  use  available  computer  resources.  These  packages 
enable  auditors  to  give  their  computers  complex  commands  that  would 
normally  require  a  computer  programming  professional.  Within  the 
agency,  the  type  of  software  used  depends  on  the  type  of  computer.  The 
Z-120  computers  have  individual  software  packages  that  perform  a  single 
function.  Auditors  use  Wordstar  for  word  processing;  Condor  or  DBase  m 
for  data  base  construction  and  manipulation;  Lotus  1  -2-3  for  spreadsheet 
analysis;  and  HyperAccess  for  personal-computer-to-personal-computer 
communication.  Additionally,  AFAA/ DOT  and  AFAA/SI  have  designed 
special  programs  for  audit  functions  such  as  break-even  analysis,  inventory 
modeling,  random  sampling,  personal-computer-to-mainframe  interface, 
and  so  forth. 

The  Z-248  and  Z-186  personal  computers  use  an  IBM-compatible  ar¬ 
chitecture  which  Is  different  from  the  Z-120.  Nonetheless,  these  machines 
run  different  versions  of  all  the  software  available  for  the  Z-120.  Addition¬ 
ally,  AFAA  has  purchased  Enable-integrated  software  packages  so  that 
auditors  can  run  word  processing,  spreadsheet,  data  base,  and  com¬ 
munications  functions  under  a  single  program. 


Summary 

In  the  computer  hardware  area  the  agency  primarily  uses  large,  central¬ 
ized  mainframe  computers  to  support  the  requirements  of  its  centralized 
directorates  while  decentralized  personal  computer  systems  support  field 
office  requirements.  In  terms  of  software  packages,  the  agency  uses 
programs  to  perform  word  processing,  spreadsheet,  data  base  construction 
and  manipulation,  and  special  audit  functions. 

Now  that  you  have  some  background  information  on  agency  operations 
and  requirements,  chapter  5  will  focus  on  user  information  needs.  It  will 
provide  the  critical  user  input  needed  for  developing  a  useful  management 
information  system. 
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Chapter  5 

Survey  and  Analysis 

The  AFAA’s  mainframe  and  personal  computer  architecture  provide  its 
personnel  with  automated  tools  for  the  gathering,  processing,  analyzing, 
and  reporting  of  audit-related  information.  AFAA  distributed  a  survey  to 
assess  how  well  the  needs  of  personnel  were  being  met  and  to  identify  areas 
where  system  Improvements  could  be  made.  Each  auditor,  staff  member, 
supervisor,  executive,  and  support  person  was  asked  to  complete  the 
survey.  Survey  results  provided  feedback  on  the  importance  of  different 
types  of  information  and  whether  it  was  available,  timely,  and  in  a  useful 
format.  Throughout  this  chapter  refer  to  the  survey  at  appendix  B. 

Survey  Rating  System  and  Design 

In  October  1988,  AFAA/DOO  distributed  over  900  surveys  and  483 
individuals  (54  percent)  responded.  This  high  percentage  of  return  provides 
a  representative  sample  of  the  population. 

The  survey  responses  ranged  from  0  to  5  (0  to  6  in  section  3).  The  “0" 
response  provided  a  mechanism  for  people  who  did  not  wish  to  respond  in 
some  areas  or  to  answer  the  remaining  questions  without  polluting  the  data 
base.  Similarly,  a  response  of  “  1"  showed  that  the  data  was  not  used  in  the 
respondent’s  particular  job.  This  £  ’owed  people  to  answer  only  those 
questions  that  pertained  to  their  Job  areas.  Responses  of  “4"  or  higher 
reflected  a  positive  response  in  that  particular  area. 

The  survey  addressed  four  areas: 

•  information  requirements, 

•  information  availability, 

•  Information  timeliness,  and 

•  information  format. 

The  analysis  of  results  led  to  conclusions  about  the  need  for,  and  best 
method  of,  providing  MIS  support  in  a  given  area. 

In  the  area  of  information  requirements,  a  high  percentage  of  “4"  (indicat¬ 
ing  that  this  information  was  important  to  the  respondent’s  work)  or  “5” 
responses  (Indicating  that  the  information  was  critical  to  the  respondent's 
work)  shows  a  strong,  AFAA-wide  need  for  this  data.  This  distribution 
indicates  the  need  for  a  centralized  approach  to  the  collection  and  manage¬ 
ment  of  the  information.  An  area  of  information  with  a  low  percentage  of 
“4”  and  “5”  responses  indicates  the  need  for  a  decentralized  approach.  Such 
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a  system  would  provide  optimum  responsiveness  to  the  few  users  who  need 
this  information.  The  survey  did  not  specifically  address  computers  or 
automation  because  information  does  not  have  to  come  from  a  computer 
to  be  useful  to  the  auditor.  Thus.  If  Information  needs  are  met  through 
“hard  copy”  sources,  there  is  no  need  to  change  the  procedure. 

The  second  area  addressed  in  the  survey  was  that  of  information 
availability.  As  might  be  expected,  information  considered  important  or 
critical  to  an  auditor’s  job  should  be  available  without  much  effort.  Infor¬ 
mation  of  low  value  or  that  is  seldom  used  should  be  available  somewhere 
but  should  require  more  effort  on  the  part  of  the  user.  It  is  at  this  point 
that  automating  information  starts  to  play  a  role  in  the  MIS  process. 
Clearly,  information  with  a  high  degree  of  perceived  importance  that  is  not 
available  to  the  user  must  be  accessible,  and  a  centralized  system  is  usually 
the  answer.  Likewise,  information  that  is  Important  to  a  limited  number  of 
people  may  still  require  automation,  and  a  decentralized  system  usually 
provides  the  solution. 

Information  timeliness,  the  third  area,  more  directly  influences  the  need 
for  automation.  Information  that  is  important  to  many  people  must  be 
available  in  time  for  it  to  be  effective.  Automation  can  greatly  enhance  the 
timeliness  of  information  availability  by  providing  on-line  access  via  remote 
computer  terminals.  Information  that  is  not  time-sensitive  could  be 
provided  using  manual  methods. 

The  final  area  addressed  in  the  survey  was  that  of  information  format. 
More  than  the  other  three  factors,  the  amount  of  automation  used  impacts 
the  formatting  of  data.  One  of  the  advantages  of  computerization  is  the 
capability  to  “customize”  output  for  individual  users.  Thus,  for  example, 
information  that  is  important  to  a  large  number  of  people  could  be  collected 
and  maintained  on  a  centralized  mainframe  system.  However,  if  the  users 
preferred  a  different  format,  they  could  download  the  information  to  a 
decentralized  personal  computer  and  reformat  it  using  application  software. 


Analysis 

For  my  preliminary  analysis  of  the  responses,  I  used  Condor  data  base 
management  software.  For  a  more  detailed  analysis  and  graphic  output  of 
the  results.  I  entered  the  Condor  retrievals  into  a  Lotus  1  -2-3  spreadsheet. 
An  analysis  of  each  of  the  25  que^  :  ions  follows. 

Research  Question  #1 — How  much  (or  how  little)  do  you  need  access  to 
current  regulations  and  directives? 

Since  compliance  with  regulations  is  often  an  Integral  part  of  auditing, 
one  would  expect  auditors  to  consider  access  to  current  regulations  and 
directives  Just  as  important  to  their  work.  Over  85  percent  of  those 
responding  felt  that  this  information  was  either  important  or  critical  to  job 
accomplishment.  Although  agency  access  to  regulations  and  directives  is 
not  automated,  AFAA  personnel  seemed  satisfied  with  its  current 
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availability.  Nearly  80  percent  rated  this  Item  as  being  reasonably  available, 
and  over  80  percent  felt  the  Information  was  “generally  current  and  timely.” 
Finally,  over  72  percent  indicated  that  the  information  was  clear  and  easy 
to  understand.  Figure  2  shows  the  strong  correlation  between  the  perceived 
importance  of  this  information  and  its  availability,  format,  and  timeliness. 
(Note:  Percentage  figures  in  the  text  are  total  averages  of  the  individual 
response  percentages  depicted  in  figures  2  through  26.) 


Figure  2.  Current  Regulations  and  Directives. 

Audit  agency  personnel  are  satisfied  with  their  access  to  current  regula¬ 
tions  and  directives.  Although  some  improvement  (i.e. .  from  good  to  perfect) 
is  possible,  any  effort  to  provide  an  automated  AFAA  system  probably  would 
not  be  warranted.  The  possibility  does  exist  for  other  organizations  to 
develop  automated  systems  for  storing,  processing,  and  distributing  regula¬ 
tions  and  directives.  When  these  systems  become  available.  AFAA  should 
obtain  “read-only"  access  to  them.  This  would  enable  audit  offices 
throughout  the  world  to  have  up-to-the-minute  access  via  dial-up  modems. 
Also  it  would  permit  auditors  to  do  automated  searches  for  specific  items 
of  audit  interest. 

Research  Question  #2 — How  much  (or  how  little)  do  you  need  access  to 
obsolete  regulations  and  directives? 

Although  information  of  this  sort  is  sometimes  useful  in  specific  audits, 
auditors  deal  primarily  with  current  directives.  The  survey  results  support 
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this  lack  of  need  as  only  15  percent  felt  it  was  valuable  information  (“4”  or 
“5").  There  were  correspondingly  low  ratings  in  the  areas  of  availability  ( 1 1 
percent),  timeliness  (12  percent),  and  format  (37  percent).  This  is  a  positive 
indicator  that  unnecessary  effort  is  not  being  expended  to  obtain  informa¬ 
tion  of  marginal  value.  Figure  3  shows  the  interrelationships  among  the 
four  factors. 
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Figure  3.  Obsolete  Regulations  and  Directives. 

The  perceived  low  value  of  this  data  does  not  justify  a  centralized  system 
to  access,  store,  or  process  obsolete  regulations.  Thus,  a  decentralized 
system  could  satisfy  any  individual  or  local  requirement.  As  mentioned  in 
question  #1,  if  future  automated  systems  provide  this  information.  AFAA 
should  consider  gaining  access. 

Research  Question  #3 — How  much  (or  how  little)  do  you  need  access  to 
technical  orders  (TOs)? 

Although  this  information  does  not  appear  to  be  valuable  to  anyone 
outside  the  using  organization,  certain  auditors  might  need  to  use  TOs  for 
operational  reviews.  The  survey  revealed  that  55  percent  felt  that  this 
information  was  Important.  Most  indicated  that  availability  (50  percent), 
timeliness  (53  percent),  and  format  (46  percent)  were  proportional  to  the 
value  of  the  information  required  (fig.  4). 

Again,  the  needs  of  the  agency  are  being  met  by  current  methods. 
Therefore,  there  is  not  a  need  for  a  centralized  system  to  maintain  TO  data. 
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However,  access  to  non-AFAA  data  systems  such  as  the  automated  techni¬ 
cal  order  system  (ATOS)  would  provide  this  data  to  auditors  with  increased 
availability  and  timeliness. 
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Figure  4.  Technical  Orders. 

Research  Question  #4 — How  much  (or  how  little)  do  you  need  access  to 
legal  references? 

Frequently,  audit  projects  require  auditors  to  be  familiar  with  laws  and 
statutes  in  a  particular  area.  From  the  survey  over  48  percent  felt  that  this 
Information  was  important  to  their  work;  35  percent  were  satisfied  with  Its 
availability;  30  percent  with  its  timeliness;  and  22  percent  with  Its  format 
(fig.  5). 

Although  the  respondents  felt  that  legal  information  is  important,  its 
access  poses  a  problem.  Therefore,  to  gain  this  access.  AFAA  should  use 
other  centralized  systems  such  as  Federal  Legal  Information  Through 
Electronics  (FLITE)  to  meet  this  need.  AFAA  personal  computers  could 
access  these  centralized  data  bases  using  existing  software.  Once  the 
agency  arranges  this  access,  they  should  provide  auditors  with  instructions 
on  how  to  use  the  system. 

Research  Question  #5 — How  much  (or  how  little)  do  you  need  access  to 
“library- type"  references? 

Occasionally,  auditors  review  areas  requiring  specific  technical 
knowledge.  From  the  survey  only  19  percent  expressed  a  significant  need 
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Figure  5.  Legal  References. 

for  this  Information;  33  percent  were  satisfied  with  its  availability;  29 
percent  with  its  timeliness;  and  45  percent  with  its  format  (fig.  6). 

Improving  the  availability  of  this  data  appears  to  be  of  marginal  value. 
Suggest  the  agency  take  no  action  in  this  area. 

Research  Question  #6 — How  much  (or  how  little)  do  you  need  access  to 
AFAA  audit  guides? 

Audit  guides  provide  auditors  with  standardized  techniques  and  tests 
that  apply  to  specific  audit  subject  areas  (e.g.,  base  supply).  Currently  they 
are  centrally  developed  and  issued  in  hard  copy  to  AFAA  offices.  Over  60 
percent  of  the  respondents  rated  audit  guides  as  important  information; 
however,  they  appeared  to  be  satisfied  with  Its  availability  (80  percent), 
timeliness  (54  percent),  and  format  (67  percent)  (fig.  7). 

The  current  method  of  providing  AFAA  audit  guides  is  working  well.  It 
does  not  appear  that  automating  this  process  would  be  cost-effective. 

Research  Question  it  7 — How  much  (or  how  little)  do  you  need  access  to 
AFAA- produced  audit  programs? 

An  auditor  develops  a  program  for  each  audit  project;  and  because  many 
audit  topics  are  similar,  programs  from  one  project  can  be  helpful  in 
designing  other  programs.  Over  47  percent  of  those  responding  rated  these 
programs  as  being  important  to  their  audits.  Auditors  were  happy  with  the 
availability  (61  percent),  timeliness  (41  percent),  and  format  (60  percent)  of 
the  information  (fig.  8). 
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Figure  6  Ltorary  References 
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Figure  7.  Air  Force  Audit  Agency  Audit  Guides 
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Figure  8.  Air  Force  Audit  Agency  Audit  Programs 

Current  methods  of  providing  audit  programs  to  auditors  are  working 
well.  Additional  development  In  this  area  would  not  be  warranted. 

Research  Question  #8 — How  much  (or  how  little)  do  you  need  the  text  of 
AFAA  local  audit  reports? 

Because  problem  areas  noted  in  one  report  of  audit  may  exist  at  another 
base  or  location,  many  auditors  use  existing  reports  to  channel  their  audit 
efforts.  Over  39  percent  rated  access  to  AFAA  local  audit  reports  as 
important  or  critical  to  their  Jobs;  however,  an  even  greater  number  were 
pleased  with  its  availability  (58  percent),  format  (59  percent),  and  timeliness 
(34  percent)  (fig.  9). 

Current  methods  for  obtaining  local  reports  of  audit  are  working  well.  It 
appears  that  further  automation  would  not  provide  appreciable  improve¬ 
ments. 

Research  Question  # 9 — How  much  (or  how  little)  do  you  need  the  text  of 
AFAA  "blue-book"  reports? 

Audit  managers  in  AFAA/FS  or  AFAA/QL  usually  produce  these  reports 
and  send  them  to  the  secretary  of  the  Air  Force.  Because  these  reports 
discuss  major  Air  Force-wide  problems  and  can  impact  on  policies  at  all 
levels,  they  are  of  Interest  to  audit  personnel  throughout  the  agency. 
Additionally,  auditors  at  base  level  can  often  adapt  the  audit  approach  in 
the  blue-book  reports  to  produce  timely  audits  at  their  particular  locations. 
Over  29  percent  of  those  responding  felt  that  these  reports  were  important 
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Figure  9.  Air  Force  Audit  Agency  Local  Reports 

or  critical  to  their  Jobs.  They  were  generally  positive  in  their  appraisal  of 
information  availability  (39  percent),  timeliness  (31  percent),  and  format 
(45  percent)  (fig.  10). 

The  current  process  of  distributing  hard  copies  of  blue-book  reports  of 
audit  to  field  offices  and  staff  agencies  meets  the  needs  of  AFAA  personnel. 
Automation  of  this  process  would  not  be  advisable  at  this  time. 

Research  Question  #10 — How  much  (or  how  little)  do  you  need  data 
related  to  AFAA  reports  of  audit? 

Auditors  use  information  such  as  the  number  of  findings,  report  title,  and 
so  forth  for  researching  potential  audit  topics.  AFAt  i  headquarters  uses  the 
information  primarily  to  respond  to  various  non-AFAA  tasklngs.  For 
example,  the  agency  must  forward  specific  report  information  to  the  DOD1G 
on  a  semiannual  basis.  This  data  is  of  moderate  value  (30  percent)  to  the 
agency  as  a  whole,  although  a  small  but  significant  number  of  respondents 
(8  percent)  felt  that  it  was  critical  to  their  Jobs.  Respondents  were  pleased 
with  the  information's  availability  (38  percent),  timeliness  (31  percent),  and 
format  (45  percent)  (fig.  11). 

The  results  of  this  question  point  out  the  need  to  combine  the  best 
features  of  both  centralized  and  decentralized  systems.  To  ease  the  uniform 
collection  of  data  from  AFAA  offices  around  the  world,  you  need  a  centralized 
system.  Because  the  use  of  the  data  tends  to  be  highly  specialized  and  the 
reporting  requirements  of  the  DODIG  and  other  external  agencies  are 
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subject  to  changes  beyond  the  control  of  the  agency,  the  formatting  power 
of  a  decentralized  system  is  more  valuable.  As  a  result.  AFAA  should  use 
standard  processes  to  collect  and  store  the  information  on  the  Wang 
mainframe  system.  Also,  AFAA  should  maintain  interfaces  that  allow  the 
data  to  be  downloaded  to  decentralized  systems  for  specialized  processing. 

Research  Question  #1 1 — How  much  (or  how  little)  do  you  need  access  to 
non-AFAA  audit  guides  and  audit  programs? 

These  non-AFAA  products  are  similar  to  AFAA  audit  guides  and 
programs,  except  that  they  are  from  other  audit  organizations.  Some 
auditors  find  the  Information  useful  in  specific  circumstances.  The  value 
of  non-AFAA  products  is  limited  because  these  reports  rarely  discuss  Air 
Force  procedures.  Only  15  percent  of  those  polled  felt  that  access  to  these 
guides  and  programs  was  important  to  their  duties.  A  corresponding 
number  of  respondents  were  satisfied  with  its  availability  (11  percent), 
timeliness  (13  percent),  and  format  (22  percent)  (fig.  12). 

This  is  a  prime  example  of  a  system  that  works  well  without  placing  any 
burden  on  the  information  system  capability  of  the  agency.  Non-AFAA 
agencies  are  providing  the  amount  of  information  that  AFAA  auditors 
require  satisfactorily.  Any  effort  to  change  current  procedures  would  be 
nonproductive. 


Figure  12  Non-Air  Force  Audit  Agency  Audit  Guides. 
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Research  Question  it  12 — How  much  (or  how  little)  do  you  need  non-AFAA 
reports  of  audit? 

Auditors  sometimes  use  non-AFAA  reports  of  audit  in  researching  an 
audit  topic.  Similar  to  non-AFAA  audit  guides  and  programs,  these  reports 
rarely  deal  directly  with  Air  Force-related  topics.  As  a  result,  only  13 
percent  of  the  respondents  thought  that  this  information  was  important  to 
their  jobs,  and  they  were  satisfied  with  its  availability  (13  percent) .  timeli¬ 
ness  (12  percent),  and  format  (28  percent)  (fig.  13). 


Figure  13.  Non-Air  Force  Audit  Agency  Audit  Reports. 

Current  methods  are  adequate  to  meet  agency  needs.  Further  effort  in 
this  area  is  unwarranted. 

Research  Question  it  13 — How  much  (or  how  little)  do  you  need  access  to 
data  related  to  non-AFAA  audits? 

Such  data  can  be  useful  for  comparing  AFAA  to  other  audit  agencies,  but 
it  would  have  little  value  beyond  that.  A  small  number  of  those  responding 
(7  percent)  felt  an  important  need  for  this  information.  Those  that  did  need 
the  data  were  pleased  with  its  availability  (6  percent),  timeliness  (7  percent), 
and  format  (16  percent)  (fig.  14). 

The  current  system  works  well.  There  is  no  need  to  change  anything. 

Research  Question  it  1 4 — How  much  (or  how  little)  do  you  need  access  to 
non-AFAA  Information  systems? 
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Figure  14.  Non-Air  Force  Audit  Agency  Audit  Data 

Similar  to  questions  #11  through  #13.  this  information  is  useful  to  certain 
individuals.  Auditors  ranked  this  Information  as  having  limited  value  (16 
percent).  Only  9  percent  of  the  respondents  gave  a  correspondingly  high 
rating  to  the  availability  of  the  data,  and  only  10  percent  gave  high  marks 
to  its  timeliness.  Under  format.  17  percent  indicated  a  favorable  response 
(fig.  15). 

The  disparity  between  the  need  for  this  data  and  its  availability  and 
timeliness  Indicates  that  improvements  are  required.  However,  the  low 
rating  under  need  suggests  that  a  decentralized  system  is  probably  the 
answer.  To  improve  both  timeliness  and  accessibility,  AFAA  auditors  and 
audit  offices  should  obtain  permanent,  “read-only”  system  access.  This 
would  give  individuals  the  capability  to  access  such  systems  at  any  time, 
using  their  decentralized  personal  computer  capability,  thus  eliminating 
the  overhead  of  a  centralized  tie-in  to  non-AFAA  systems. 

Research  Question  #  1 5 — How  much  (or  how  little)  do  you  need  access  to 
classified  Information? 

Because  the  AFAA  audit  mission  involves  the  entire  spectrum  of  Air  Force 
activities,  auditors  may  require  access  to  classified  data.  In  fact,  35  percent 
of  all  respondents  felt  that  classified  information  was  important  or  critical 
to  their  jobs.  Unlike  the  situation  noted  in  question  #14,  audit  personnel 
found  that  the  availability  (32  percent),  timeliness  (39  percent),  and  format 
(40  percent)  of  classified  information  corresponded  to  its  importance  (fig. 
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16).  Although  the  agency  has  no  difficulty  In  obtaining  access  to  all  levels 
of  classified  data.  It  has  problems  in  accessing  non-AFAA  information 
systems.  This  is  probably  due  to  two  factors:  well-established  procedures 
for  obtaining  access  to  classified  information  and  preauthorized  access  to 
classified  information.  Audit  personnel  have  valid  security  clearances  and 
can  easily  gain  access  to  this  information  as  soon  as  they  establish  a  need 
to  know. 

Current  procedures  for  obtaining  classified  information  provide  ade¬ 
quate.  timely,  and  properly  formatted  information  for  audit  functions.  The 
agency  should  adopt  similar  procedures  to  obtain  access  to  non-AFAA 
information  systems. 

Research  Questions  # 16-22 — These  specific  items  of  information  relate 
to  the  management  of  audit  resources.  The  questions  refer  to  the  need  for 
data  during  project  management  by  auditors,  supervisors,  audit  managers, 
executives,  audit  clerks,  and  others. 

Research  Question  it  16 — How  much  (or  how  little)  do  you  need  informa¬ 
tion  about  ongoing  audit  projects? 

Knowing,  for  example,  how  many  man-days  have  been  invested  in  an 
audit  effort  helps  audit  supervisors  to  manage  their  work  loads.  Currently, 
the  centralized  Wang  mainframe  system  collects  and  maintains  this  data. 
A  large  number  of  respondents  (44  percent)  ranked  this  information  as 
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Figure  t6.  Classified  Information. 


Important  to  their  audit  duties.  Also,  they  gave  high  marks  for  availability 
(55  percent),  timeliness  (39  percent),  and  format  (51  percent)  (fig.  17). 

The  centralized  system  for  obtaining,  processing,  and  distributing  this 
information  is  appropriate  and  is  providing  satisfactory  results.  The  agency 
could  Improve  timeliness  by  automating  the  information  flow  from  field 
offices  to  the  centralized  computer. 

Research  Question  #1 7 — How  much  (or  how  little)  do  you  need  informa¬ 
tion  about  planned  audit  projects? 

Information  such  as  this  is  Important  to  auditors  and  their  supervisors 
because  it  facilitates  an  orderly  and  efficient  work  flow.  Over  39  percent  of 
the  respondents  rated  this  as  important  or  critical  to  their  jobs,  and  they 
gave  high  ratings  for  availability  (55  percent),  timeliness  (37  percent),  and 
format  (47  percent)  (fig.  18). 

Users  felt  that  current  methods  were  satisfactory,  with  no  problem  in 
terms  of  availability,  timeliness,  and  format.  Increased  effort  in  this  area 
would  not  be  productive  at  this  time. 

Research  Question  It  18 — How  much  (or  how  little)  do  you  need  access  to 
summary  data? 

AFAA  headquarters  personnel  would  find  this  Information  especially 
important  because  it  could  provide  office  productivity  comparisons  and 
performance  trends.  Respondents  rated  this  information  as  average  impor¬ 
tance,  with  29  percent  feeling  that  it  was  Important  or  critical  to  their 


39 


1 


2 


RATING 


3 


|flH!il  AVAILABILITY 


TIMELINESS 


|  |  FORMAT 


Figure  17.  Ongoing  Projects. 
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Figure  18  Planned  Projects 
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mission.  Although  they  were  satisfied  with  its  availability  (31  percent)  and 
format  (24  percent),  the  timeliness  of  the  information  (19  percent)  did  not 
quite  match  their  requirements  (fig.  19).  Of  those  who  felt  that  this 
information  was  critical  (7  percent),  availability  and  format  percentages 
were  much  lower.  Only  2  percent  indicated  that  the  information  was 
current  and  on  time.  Less  than  0.4  percent  characterized  the  format  as 
“Exactly  the  format  I  want." 


Figure  19.  Summary  Data. 

Because  the  information's  availability  was  rated  higher  than  its  impor¬ 
tance.  we  can  say  that  current  centralized  methods  are  working  well.  The 
relative  lack  of  timeliness  and  adequate  format  points  out  the  need  to 
strengthen  our  decentralized  ability  to  retrieve  and  process  the  Information. 
There  are  two  solutions  to  this  problem.  The  first  solution  is  to  use 
automated  interfaces  to  allow  users  with  decentralized  personal  computers 
to  access  the  information  at  the  centralized  computer.  Also,  decentralized 
processing  capability  is  another  solution.  This  involves  using  the  data  base 
capability  of  Enable  (the  AFAA  standard  application  software)  to  allow  users 
to  customize  report  output  to  their  individual  tastes. 

Research  Question  #  1 9 — How  much  (or  how  little)  do  you  need  access  to 
productivity  trends? 

Such  data  allows  managers  and  supervisors  to  identify  trends  in  impor¬ 
tant  statistical  areas  and  to  correct  those  that  are  unsatisfactory.  Only  13 
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percent  of  the  respondents  felt  that  such  data  was  relatively  important,  with 
its  availability  (16  percent),  timeliness  (13  percent),  and  format  (18  percent) 
proportional  to  its  perceived  importance  (fig.  20). 

Overall,  the  need  for  this  information  is  being  met  by  current  procedures. 
There  is  no  need  for  additional  effort  in  this  area. 


100 


90 

80 


1 

a 

s 

8> 

Ul 

cc 

£ 

LU 

O 

CC 


NEED  llllllli  AVAILABILITY  H  T1WUNESS  \  \  FORMAT 

Figure  20.  Office  Trends 

Research  Question  # 20 — How  much  (or  how  little)  do  you  need  informa¬ 
tion  about  the  area  to  be  audited? 

Information  concerning  the  size  and  mission  of  an  audit  entity,  its  relative 
importance  to  the  overall  mission,  and  the  date  of  the  last  audit  in  the  area 
would  be  useful  to  auditors  and  their  immediate  supervisors.  They  could 
concentrate  their  efforts  on  the  most  Important  areas  and  avoid  overaudit¬ 
ing  smaller  clients.  Not  surprisingly,  almost  55  percent  of  AFAA  personnel 
thought  that  this  information  was  important  or  critical.  Unfortunately,  its 
availability  (31  percent),  timeliness  (33  percent),  and  format  (43  percent) 
were  unsatisfactory  (fig.  21).  For  those  who  indicated  that  this  information 
was  critical  (16  percent),  the  gap  was  even  greater.  Only  7  percent  thought 
the  information  was  “easily  available,"  with  less  than  4  percent  pleased  with 
its  timeliness  and  less  than  1  percent  with  its  format. 

Because  a  high  percentage  of  respondents  rated  this  information  as 
important,  a  centralized  system  is  necessary.  However,  the  need  for 
timeliness  and  format  indicates  that  decentralized  accessing  and  process- 
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ing  of  the  information  would  be  useful.  Thus,  the  agency  should  use  its 
mainframe  computers  to  provide  a  centralized  repository  for  this  data.  The 
agency  should  also  permit  automated  access  to  the  data  via  communica¬ 
tions  software  and  provide  end  users  with  the  decentralized  capability  to 
process  the  data  in  the  desired  format. 


Figure  21 .  Audit  Entity  Information. 

Research  Question  #21 — How  much  (or  how  little)  do  you  need  informa¬ 
tion  on  audit  nonconcurrences? 

Nonconcurrences  arise  when  a  client  disagrees  with  the  contents  of  an 
audit  report.  If  this  occurs,  both  sides  elevate  the  dispute  to  a  higher  level 
for  adjudication.  Therefore,  knowing  the  status  of  nonconcurrences  helps 
to  resolve  the  matter  faster.  Slightly  over  32  percent  felt  that  information 
about  nonconcurrences  was  very  important  to  their  jobs,  and  they  were 
pleased  with  its  availability  (36  percent)  and  format  (34  percent).  Only  24 
percent  indicated  that  this  information  was  timely  (fig.  22). 

Because  a  moderate  number  of  respondents  ranked  this  information  as 
important  to  their  audit  mission,  a  centralized  system  is  suggested.  Also, 
because  the  only  area  of  concern  was  timeliness,  automated  communica¬ 
tion  between  the  user  and  the  centralized  system  is  recommended.  There 
is  no  need  for  the  user  to  reformat  the  data. 

Research  Question  #22 — How  much  (or  how  little)  do  you  need  informa¬ 
tion  about  potential  monetaiy  benefits? 
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Figure  22.  Audit  Nonconcurrence  Data. 

When  an  audit  uncovers  opportunities  to  save  resources,  the  auditor  and 
the  client  estimate  the  possible  savings  and  forward  them  as  part  of  the 
report.  Subsequently,  auditors  track  these  savings  to  determine  if  they 
were  obtained  and  to  verify  the  accuracy  of  the  estimate.  From  the  survey 
over  26  percent  felt  that  this  information  was  very  important  to  their  jobs; 
with  3 1  percent  satisfied  with  its  availability;  and  32  percent  with  its  format. 
As  in  question  #21,  timeliness  (19  percent)  was  a  problem  (fig.  23). 

Because  a  moderate  number  of  respondents  ranked  this  information  as 
important  to  their  job,  a  centralized  system  is  needed.  Because  timeliness 
is  obviously  a  problem,  automated  communication  between  the  user  and 
this  system  is  recommended.  However,  the  user  will  not  need  to  reformat 
the  data. 

Research  Questions  #23-25 — This  section  of  the  survey  solicited  input  on 
requirements  for  administrative  data.  These  questions  referred  to  the  need 
for  certain  nonaudit  data  essential  for  the  administration  of  AFAA  offices 
and  directorates. 

Research  Question  # 23 — How  much  (or  how  little)  do  you  need  access  to 
information  for  civilian  and  military  appraisals? 

Supervisors  require  this  Information  for  writing  periodic  performance 
appraisals  on  their  subordinates.  Thus,  slightly  over  28  percent  thought 
that  such  information  was  either  Important  or  critical.  While  they  gave 
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Figure  23.  Potential  Monetary  Benefits  Data 

higher  marks  for  availability  (31  percent),  timeliness  received  25  percent, 
and  format  27  percent  (fig.  24). 

Users  felt  that  current  methods  were  satisfactory.  Increased  effort  in  this 
area  is  not  recommended  at  this  time. 

Research  Question  #24 — How  much  (or  how  little)  do  you  need  access  to 
leave  and  timekeeping  information? 

Supervisors  use  this  information  to  ensure  the  accuracy  and  integrity  of 
the  leave  and  timekeeping  systems.  Nearly  28  percent  of  the  respondents 
felt  that  this  information  was  important.  Most  indicated  that  availability 
(40  percent),  timeliness  (40  percent),  and  format  (46  percent)  were  propor¬ 
tional  to  the  value  of  the  information  (fig.  25). 

Current  methods  are  meeting  agency  needs.  Additional  effort  in  tills  area 
is  unwarranted. 

Research  Question  #25 — How  much  (or  how  little)  do  you  need  access  to 
employee  training  information? 

Over  39  percent  felt  that  access  to  this  information  was  very  valuable. 
Nearly  40  percent  liked  its  availability.  36  percent  its  timeliness,  and  41 
percent  its  format  (fig.  26). 

Current  methods  of  tracking  and  obtaining  employee  training  data  are 
effective.  No  additional  effort  is  necessary. 
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Figure  26.  Employee  Training  Information. 


Summary 

This  chapter  provided  the  results  of  an  Air  Force  Audit  Agency  survey 
about  access  to  audit-related  Information  and  Its  current  availability, 
timeliness,  and  format.  For  the  most  part,  the  current  mixture  of  central¬ 
ized  and  decentralized  information  systems  is  providing  agency  personnel 
with  the  Information  they  need.  Also,  the  agency  is  not  expending  needless 
effort  and  resources  In  providing  information  that  only  a  few  people  desire. 
Where  appropriate.  1  recommended  specific  areas  in  which  better  access  to 
Information  is  needed. 

The  final  chapter  provides  a  management  approach  for  the  development 
of  the  information  system.  Also,  it  identifies  the  next  step  In  this  process. 


47 


Chapter  6 


Concluding  Remarks 

The  current  MIS  capabilities  of  AFAA  consist  of  a  combination  of  central¬ 
ized  systems  that  run  on  the  Wang  mainframe  system  at  Norton  AFB  and 
Independently  developed  decentralized  systems  that  run  on  personal  com¬ 
puters  at  various  directorates,  divisions,  and  offices.  Although  there  are 
technical  experts  at  the  AFAA  Small  Computer  Technical  Center. 
AFAA/DOT,  and  AFAA/SI.  there  is  no  central  manager  to  oversee  the 
agency’s  development  of  its  information  system. 


Recommendations 

To  provide  a  central  direction  for  agency  MIS  efforts.  I  recommend  the 
following  actions: 

1.  Establish  a  position  for  an  information  system  manager.  This 
Individual  would  monitor  and  oversee  the  development,  maintenance,  and 
use  of  agency  information  systems.  The  position  should  be  at  a  civilian 
grade  high  enough  to  ensure  agencywide  compliance  in  the  development  of 
an  Integrated  MIS.  consisting  of  both  centralized  and  decentralized  systems. 
The  individual  should  be  a  professional  auditor  who  has  headquarters  and 
field  experience. 

2.  Direct  the  information  system  manager  to  perform  several  functions. 
Ftrst,  define  those  areas  of  automation  that  individuals  can  develop  on  their 
own.  These  “individual  applications"  would  be  onetime,  short-duration 
computer  projects  that  meet  a  nonrecurring  requirement .  For  example,  an 
auditor  who  needs  to  construct  a  spreadsheet  or  small  data  base  for  a  single 
audit  project  would  not  be  required  to  consult  this  manager. 

Second,  advise  decentralized  system  users  on  interfacing,  software,  and 
downloading.  Thus,  the  information  system  manager  should  establish 
standards  for  the  identification  of  MIS  requirements,  the  design  of  systems 
to  meet  those  requirements,  the  required  documenlation  for  the  proposed 
systems,  and  the  requirements  to  maintain  a  system  once  It  is  established. 
The  goal  here  is  not  to  discourage  decentralized  development  of  these 
systems,  but  to  ensure  that  they  can  Interface  with  other  agency  systems, 
use  standard  software,  and  are  adequately  documented.  Third,  with  the 
Defense  Data  Network  (DDN)  becoming  available  In  the  near  future,  the 
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Information  system  manager  should  oversee  the  use  of  DDN  capabilities 
(plus  any  other  methodology)  to  interconnect  AFAA  offices  and  directorates 
throughout  the  world. 

3.  Establish  a  position  for  a  computer  specialist.  This  individual  would 
advise  the  information  manager  in  directing  information  systems'  efforts. 
Also,  this  person  should  possess  sufficient  grade  and  experience  to  interact 
with  AFAA  personnel  at  all  levels. 

4.  Direct  the  computer  specialist  to  develop  an  AFAA-wide  data  diction¬ 
ary.  It  would  describe  the  format  of  each  data  item  maintained  in  the 
agency’s  information  systems.  This  dictionary  would  also  designate  offices 
of  primary  responsibility  (OPR)  for  each  data  item.  When  practical,  offices 
or  personnel  who  wanted  to  develop  an  additional  decentralized  system 
could  access  and  use  data  from  the  OPR’s  system,  rather  than  duplicating 
information  which  is  already  available. 


Concluding  Remarks 

AFAA  has  already  made  good  progress  in  ensuring  that  its  personnel  have 
access  to  the  information  required  to  do  their  Jobs.  In  fact,  audit  personnel 
at  all  levels  are.  with  few  exceptions,  pleased  with  its  availability,  timeliness, 
and  format.  With  centralized  management  and  unit  cooperation,  this  MIS 
effort  will  provide  better  service  to  the  user,  resulting  in  Improved  audit 
products. 
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Air  Force  Audit  Agency  Hardware 
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Air  Force  Audit  Agency  Hardware 
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5 

0 

1 

0 

0 

0 

0 

0 

6 

12 

200 

6 

200 

Directorate  of  Resource  Management 

TOTAL 

20 

0 

4 

1 

7 

0 

0 

0 

32 

44 

1  38 

24 

1  83 

54 


Air  Force  Audit  Agency  Hardware 


DATE  REVISED  08-Mar-88  AIR  FORCE  AUDIT  AGENCY  (AFAA)ADPE  FILENAME:  AFAAADPE 


Number  of 
Z  248  and 


Z248 

Z120 

Other  #ot 

Pars 

Laptops 

Pars 

Hard 

Z  184 

Hard 

Tempest 

Portable  Other  Total  Auth 

per 

IBM 

per 

LOCATION 

Disk  Z  248 

Laptops 

Disk 

Z120  Micros 

o 

a 

o 

i 

Computer 

Compat 

Compat 

Director  and  Stall 

4 

0 

3 

0 

0 

0 

0 

0 

7 

5 

0.71 

7 

0.71 

Automated  Systems  Division 

7 

0 

15 

0 

0 

0 

4 

0 

26 

23 

088 

26 

088 

Comptroller  Audits  Division 

4 

0 

16 

0 

0 

0 

0 

0 

20 

23 

1.15 

20 

1  15 

Personnel  Audrts  Division 

5 

0 

13 

0 

0 

0 

1 

0 

19 

18 

095 

19 

095 

Support  Services  Division 

2 

0 

13 

0 

0 

2 

1 

0 

18 

18 

100 

18 

100 

HQ  FS  TOTAL  (Headquarters) 

22 

0 

60 

0 

0 

2 

6 

0 

90 

87 

0.97 

90 

0.97 

Training  Audit  Division 

(Rando^jh  AFB,  TX) 

0 

0 

1 

0 

0 

0 

0 

0 

1 

3 

3,00 

1 

300 

Gunter  AFB,  AL 

3 

0 

5 

1 

2 

0 

0 

0 

11 

17 

1.55 

8 

2  13 

KeeslerAFB,  MS 

2 

0 

1 

0 

1 

0 

0 

0 

4 

6 

1  50 

3 

2.00 

Lowry  AFB,  CO 

3 

0 

1 

1 

1 

0 

1 

0 

7 

19 

2.71 

5 

380 

Mather  AFB,  CA 

1 

0 

0 

0 

1 

0 

0 

0 

2 

6 

300 

1 

600 

Randolph  AFB,  TX 

4 

0 

1 

0 

2 

0 

0 

0 

7 

IB 

2.57 

5 

360 

Sheppard  AFB.  TX 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

FST  TOTAL  (Field  Units) 

14 

0 

10 

2 

8 

0 

1 

0 

35 

75 

2.14 

25 

3.00 

Finance  and  Support  Directorate 


TOTAL 

36 

0 

70 

2 

8 

2 

7 

0 

125  162  1  30 

115  1.41 

55 


1 


Air  Force  Audit  Agency  Hardware 

DATE  REVISED  08-Mar-88  AIR  FORCE  AUDIT  AGENCY  (AFAA)  ADPE  FILE  NAME:  AFAAADPE 


Number  ot 
Z  248  and 


Z  248 

Z120 

Other  dot 

Pers 

Laptops 

Pers 

Hard 

Z  184 

Hard 

Tempest 

Portable  Other  Total  Auth 

per 

IBM 

per 

LOCATION 

Disk 

Z  248  Laptops 

Disk 

Z120  Micros 

Computers  WP  Computers  Pers 

Computer 

Compat 

Compat 

Director  and  STaff 

15 

2 

42 

0 

5 

2 

3 

0 

89 

100 

1  45 

64 

1  56 

Logistics  Audits 

(Wright  Patterson  APB  OH) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4 

000 

0 

000 

HilAFB.UT 

3 

0 

5 

1 

0 

0 

0 

0 

9 

21 

2  33 

8 

263 

Kelly  AFB.TX 

3 

0 

6 

1 

2 

0 

0 

0 

12 

22 

1  83 

9 

2  44 

McClellan  APB.  CA 

3 

0 

5 

1 

4 

0 

0 

0 

13 

20 

1  54 

8 

250 

Rob  ns  AFB.GA 

5 

1 

8 

1 

1 

0 

0 

0 

16 

22 

1  38 

14 

1  57 

Tinker  APB,  OK 

4 

0 

7 

0 

2 

0 

0 

0 

13 

20 

1.54 

11 

1  82 

WnghtPalterson  APB,  OH 

5 

0 

2 

0 

3 

1 

0 

1 

12 

7 

0.58 

8 

088 

QU.  TOTAL 

(Logistics  Division) 

23 

1 

33 

4 

12 

1 

0 

1 

75 

116 

1  55 

58 

200 

Acqtisttion  Audits 

(Andrews  AFB.MD) 

2 

0 

1 

1 

1 

0 

0 

0 

5 

5 

1  00 

3 

1  67 

Andrews  AFB.  MD 

2 

1 

3 

0 

2 

0 

0 

0 

8 

14 

1.75 

6 

233 

ASD  Wnght  Patterson,  OH 

0 

0 

5 

0 

0 

0 

0 

0 

5 

22 

4  40 

5 

440 

Edwards  AFB,  CA 

1 

0 

1 

0 

2 

0 

0 

1 

5 

6 

1  20 

2 

300 

EglnAFB,  FL 

1 

3 

2 

2 

1 

0 

0 

1 

10 

21 

2  10 

6 

350 

Kirtland  AFB,  NM 

2 

0 

2 

0 

1 

0 

0 

0 

5 

6 

1  20 

4 

1  50 

BMCVNorton  APB.  CA 

0 

0 

2 

0 

0 

0 

0 

0 

2 

16 

800 

2 

800 

Hanscom  AFB,  MA 

2 

0 

4 

2 

0 

0 

0 

0 

8 

15 

1  88 

6 

250 

Patrick  AFB,  FL 

1 

0 

2 

0 

4 

0 

0 

0 

7 

7 

1  00 

3 

2  33 

Vandenberg  AFB,  CA 

1 

1 

1 

1 

0 

0 

0 

0 

4 

6 

1  50 

3 

200 

QLO  TOTAL 

(Acquisition  Div) 

12 

5 

23 

6 

11 

0 

0 

2 

59 

118 

200 

40 

2  95 

AcquBition  and  Logsttcs  Directorate 

TOTAL  50 

e 

98 

10 

28 

3 

3 

3 

203  334 

1  65 

162 

206 

I 
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Air  Force  Audit  Agency  Hardware 


DATE  REVISED 


08-Mar-88  AIR  FORCE  AUDIT  AGENCY  (AFAA)  ADPE  FILE  NAME:  AFAAADPE 


LOCATION 

Z-246 

Hard 

Disk 

Z-248 

Z  184 
Laptops 

Z120 

Hard 

Disk 

Z120 

Tempest 

Micros 

Other 

Portable 

Computers 

»  of 

Other  Total  Auth 

WP  Computers  Pers 

Pers 

per 

Computer 

Division  and  Staff 

m 

■ 

0 

0 

1 

0 

0 

0 

i  i 

TOO 

Reports  and  Projects  Branch 

2 

an 

0 

0 

1 

0 

0 

3  5 

1.67 

CDA  Management  Branch 

2 

0 

2 

0 

0 

0 

0 

bS 

4  e 

1.50 

Technical  Services  Division 

4 

0 

1 

6 

0 

1 

12  12 

TOO 

Standardization  Division 

1 

0 

0 

0 

1 

0 

0 

MM 

2  6 

300 

Directorate  of  Operations 

TOTAL 

9 

0 

2 

1 

9 

0 

1 

0 

22  30 

1.36 

) 


57 


Air  Force  Audit  Agency  Hardware 


DATE  REVISED  08-Mar-88  AIR  FORCE  AUDIT  AGENCY  (AFAA)  ADPE  FILE  NAME:  AFAAADPE 


Number  of 
Z  248  and 


LOCATION 

Z  248 
Hard 
Disk 

Z-248 

Z 184 
Laptops 

Z120 

Hard 

Disk 

Z120 

Tempest 

Micros 

Other  #  of 

Portable  Other  Total  A*Ah 

Computers  WP  Computers  Pars 

Pers 

per 

Computer 

Laptops 

IBM 

Compat 

Pers 

per 

Compel 

Division  and  Staff 

1 

0 

7 

1 

0 

0 

0 

0 

9 

5 

0.56 

8 

0.63 

MAC  Bases 

(Scott  AFB.IL) 

t 

0 

2 

1 

2 

0 

0 

1 

7 

3 

0.43 

3 

1.00 

Charleston  AFB,  SC 

1 

0 

1 

0 

2 

0 

0 

0 

4 

6 

1.50 

2 

3.00 

Little  Rock  AFB,  AR 

1 

0 

1 

0 

1 

0 

0 

0 

3 

7 

2,33 

2 

350 

McChord  AFB.  WA 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

300 

McGuire  AFB.  NJ 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

3.00 

Norton  AFB.  CA 

3 

0 

2 

0 

3 

0 

1 

0 

9 

6 

067 

6 

1.00 

Scott  AFB,  IL 

2 

0 

1 

0 

2 

0 

0 

0 

5 

15 

3.00 

3 

500 

Travis  AFB,  CA 

1 

0 

0 

0 

1 

0 

0 

0 

2 

6 

3.00 

1 

6.00 

TOTAL 

11 

0 

9 

1 

13 

0 

1 

1 

36 

55 

1.53 

21 

2.62 

PACAF  Bases 

(Hickam  AFB.  HI) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

000 

0 

000 

Clark  AB.  PI 

2 

0 

1 

1 

0 

0 

0 

0 

4 

7 

1.75 

3 

2.33 

Hickam  AFB,  HI 

1 

1 

1 

0 

1 

0 

0 

0 

4 

8 

2.00 

3 

2.67 

Kadena  AB,  JA 

2 

0 

7 

2 

0 

0 

1 

0 

12 

15 

1.25 

10 

150 

Misawa  AB,  JA 

1 

0 

1 

1 

0 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

Osan  AB,  KO 

2 

0 

4 

0 

2 

0 

1 

0 

9 

16 

1.78 

7 

2.29 

TOTAL 

8 

1 

14 

4 

3 

0 

2 

0 

32 

54 

1.69 

25 

2.16 

SAC  Bases 

(Offutt  AFB,  NE) 

2 

0 

1 

1 

1 

0 

0 

0 

5 

4 

0.80 

3 

1.33 

Barksdale  AFB,  LA 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

300 

Castle  AFB,  CA 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

3.00 

Dyess  AFB,  TX 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

3.00 

Ellsworth  AFB.  SD 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

Fairchild  AFB,  WA 

t 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

300 

Grand  Forks  AFB,  ND 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

3.00 

Grtffiss  AFB,  NY 

1 

0 

1 

0 

1 

0 

0 

0 

3 

7 

2.33 

2 

3.50 

K  1  Sawyer  AFB.  Ml 

1 

0 

2 

0 

2 

0 

0 

0 

5 

7 

1  40 

3 

233 

McConnell  AFB,  KS 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

300 

Minot  AFB.  ND 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

Offutt  AFB,  NE 

2 

0 

1 

0 

2 

0 

0 

0 

5 

15 

3.00 

3 

5.00 

Pease  AFB,  NH 

1 

0 

1 

0 

2 

0 

0 

0 

4 

7 

1.75 

2 

3.50 

Wurtsrmth  AFB,  Ml 

1 

0 

1 

0 

2 

0 

0 

0 

4 

6 

1.50 

2 

300 

Air  Force  Audit  Agency  Hardware 

DATE  REVISED  084lar-88  AIR  FORCE  AUDIT  AGENCY  (AFAA)  ADPE  FILE  NAME:  AFAAADPE 


Number  of 
Z-248  and 

Z-248  Z120  Other  #ol  Pers  Laptops  Pers 

H ard  Z- 184  H aid  Tempest  Portable  Other  Total  Auth  per  IBM  per 

LOCATION  Disk  Z-248  Laptops  Disk  Z120  Mfcros  Computers  WP  Computers  Pers  Computer  Compat  Compat 


1 

TACBmm 

(Langley  APB,  VA) 

2 

0 

1 

0 

0 

0 

0 

0 

3 

4 

1.33 

3 

1  33 

Bergstrom  APB,  TX 

1 

0 

1 

0 

2 

0 

0 

0 

4 

7 

1.75 

2 

3.50 

Cannon  APB,  NM 

1 

0 

1 

0 

1 

0 

0 

0 

3 

7 

2.33 

2 

3.50 

Davb-MoMhanAFB.AZ 

1 

0 

0 

0 

1 

0 

0 

0 

2 

7 

3.50 

1 

7.00 

George  APB,  CA 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

llritnmen  ADD  All 

nowwi  vtj,  raw 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

Langley  APB,  VA 

3 

1 

1 

0 

2 

0 

0 

0 

7 

15 

2.14 

5 

3.00 

LufceAFB.AZ 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

3.00 

MacCMAFB.FL 

2 

0 

2 

0 

2 

0 

0 

0 

6 

7 

1.17 

4 

1.75 

Mountain  Home  AFB,  D 

1 

1 

1 

0 

1 

0 

0 

0 

4 

6 

1.50 

3 

200 

Nets  APB,  NV 

2 

0 

2 

0 

1 

0 

0 

0 

5 

7 

1  40 

4 

1  75 

Stymour  Johnson  AFB,  NC 

1 

0 

1 

0 

1 

0 

0 

0 

3 

7 

2.33 

2 

3.50 

Shaw  APB,  SC 

1 

0 

1 

0 

1 

0 

0 

1 

4 

7 

1.75 

2 

3.50 

TOTAL 

18 

2 

14 

0 

15 

0 

0 

1 

50 

92 

1.84 

34 

2.71 

USAFE 

(RameleinAB,  QE) 

1 

0 

1 

1 

1 

0 

0 

0 

4 

4 

1  00 

2 

2.00 

RAFAteortbury.UK 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

300 

RAFBentwatom.UK 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

300 

BMwgAB.QE 

2 

0 

1 

0 

2 

0 

0 

0 

5 

7 

1.40 

3 

2.33 

RAF  Omenham  Common,  UK 

1 

0 

0 

0 

1 

0 

0 

0 

2 

6 

300 

1 

600 

Hahn  AB,  QE 

1 

1 

1 

0 

1 

0 

0 

0 

4 

6 

1.50 

3 

2.00 

RAFLakentaeti,  UK 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

2.00 

2 

300 

Rametoin  AB,  GE 

1 

0 

4 

0 

3 

0 

0 

1 

9 

22 

244 

5 

440 

TonajonAB.SP 

1 

0 

1 

0 

1 

0 

0 

0 

3 

7 

2.33 

2 

3  50 

RAF  Upper  Heytord,  UK 

1 

0 

1 

0 

1 

0 

0 

0 

3 

6 

200 

2 

300 

TOTAL 

11 

1 

12 

1 

13 

0 

0 

1 

39 

76 

1,95 

24 

3  17 

SPACECOM 

hereon  APB,  CO 

1 

0 

1 

0 

1 

0 

0 

0 

3 

8 

267 

2 

4.00 

Alaakan  Air  Command 

Bmandorf  AFB,  AK 

2 

0 

1 

3 

2 

1 

0 

1 

10 

7 

0  70 

4 

1  75 

FMAdvidnOndonti 

1  TOTAL 

SO 

2 

5w 

11 

50 

1 

3 

3 

179 

391 

2.18 

115 

340 
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DATE  REVISED 

08-Mar-88 

AIR  FORCE  AUDIT  AGENCY  (AFAA)  ADPE 

FILENAME:  AFAAADPE 

Number  ot 

Z  248  and 

Z-248 

Z120 

Other 

#of 

Pers 

Laptops 

Pers 

Hard 

Z  184 

Hard 

Tempest 

Portable 

Other 

Total  Auth 

per 

IBM 

per 

LOCATION 

Disk  Z-248 

Laptops 

Disk 

Z120  Micros 

Computers 

WP  Computers  Pers 

Computer 

Compat 

Compat 

HEADQUARTERSTOTAL 

32  0 

69 

2 

9  2 

■ 

121  166 

1.37 

110 

1.51 

GRAND  TOTAL 

145  10 

229 

24 

95  6 

14 

6 

529  882 

1.67 

404 

2.18 

60 


APPENDIX  B 


INFORMATION  REQUIREMENTS  SURVEY 


Instructions 

As  you  may  already  know,  I  have  been  assigned  to  Air  University  to  identify  AFAA 
information  requirements  at  all  levels.  The  ultimate  goal  is  to  incorporate  these 
requirements  into  the  AFAA  MIS.  This  survey  is  the  first  step  toward  identifying 
these  requirements. 

Needless  to  say,  this  is  your  opportunity  to  input  your  needs,  wants,  and  desires 
into  the  agency’s  MIS.  I  am  actively  solicit  Ing  the  opinions  of  every  AF AA-assigned 
person  in  every  directorate.  Please  take  some  time  to  fill  out  the  survey  as 
completely  as  you  can. 

Before  you  start,  however,  there  are  a  few  points  I  need  to  emphasize: 

1.  The  survey  asks  for  some  personal  data.  I  need  this  information  to  help  me 
understand  your  requirements  and  to  ask  for  future  clarification. 

2.  Please  identify  all  your  requirements  in  terms  of  the  information  or 
capabilities  you  need  to  do  your  job.  Avoid  the  temptation  to  ask  for  particular 
pieces  of  hardware  or  software.  For  example,  if  you  need  to  be  able  to  sort .  analyze, 
and  perform  computations  on  up  to  30,000  records  of  up  to  500  characters  per 
record,  then  say,  “I  need  to  sort,  analyze,  and  so  forth."  Avoid  saying,  “I  need 
DBase  III  and  a  hard  disk." 

3.  Where  appropriate,  please  include  any  reference  or  supporting  document  for 
your  requirement(s).  An  Information  requirement  that  is  mandated  by  law  or 
regulation  is  easier  to  defend  later  on. 

4.  Think  as  creatively  and  as  imaginatively  as  you  ran.  If  there  is  any 
information  or  capability  that  can  help  you  do  a  better  job.  include  it!  (Please 
understand,  however,  that  certain  requirements  may  prove  too  costly  or  labor- 
intensive  to  fulfill.) 

This  is  a  rare  opportunity  for  each  and  every  one  of  you  to  help  shape  the  future 
of  the  AFAA.  Thanks  in  advance  for  your  assistance. 


CHARLES  A.  HOBBS.  Major,  IJSAF 
AIJ  Research  Fellow 


SECTION  1— PERSONAL  INFORMATION 

PURPOSE:  This  section  will  provide  us  with  data  about  you  and  your  audit 
experience.  We  will  use  this  information  to  draw  conclusions  about  information 
requirements  for  various  classes  of  AFAA  personnel.  Personal  data  such  as  your 
name  are  requested  to  facilitate  future  contact. 

PLEASE  PROVIDE  THE  FOLLOWING: 

NAME  _ 

GRADE  _ 

OFFICE  _  AUTOVON  #  _ 

DIRECTORATE  _ 

YEARS  OF  FEDERAL  SVC  _  YEARS  IN  AFAA  _ 

CURRENT  POSITION  (AIC.  OC,  AM.  etc.)  _ 

PRIOR  AFAA  EXPERIENCE  _ 
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SECTION  2 — ASSESSMENT  OF  INFORMATION  NEEDS 


PURPOSE:  This  section  elicits  your  opinion  as  to  the  need  for  various  types  of 
information.  After  each  statement,  please  circle  the  response  which  best  describes 
your  opinion.  Use  the  following  guide  to  determine  your  response. 

0 — No  opinion.  Circle  the  number  “0"  if  you  are  not  familiar  with  the  information 
described. 

1 —  This  information  is  not  needed  at  alL  Circle  the  number  “1"  if  you  feel  the 
information  described  is  not  necessary  to  the  accomplishment  of  the  audit  mission. 

2 —  This  information  is  of  nominal  value.  Circle  the  number  “2"  if  you  feel  that  the 
information  described  has  some  negligible  value  in  accomplishing  the  audit 
mission;  however,  failure  to  have  it  would  not  impair  audit  effectiveness. 

3 —  Nice  to  have  information.  Circle  the  number  “3"  if  you  feel  that  the  information 
described  would  be  of  some  measurable  benefit  in  accomplishing  the  audit 
mission.  The  lack  of  the  information  would  make  mission  accomplishment  more 
difficult,  but  the  job  would  still  get  done. 

4 —  Important  information.  Circle  the  number  “4"  if  you  feel  that  the  information 
described  plays  an  important  role  in  the  audit  mission.  Failure  to  have  this 
information  would  make  the  audit  mission  difficult  to  accomplish. 

5 —  Critical  information.  Circle  the  number  “5”  if  you  feel  that  the  information 
described  is  indispensable.  In  other  words,  without  this  information  the  audit 
mission  could  not  be  accomplished. 

Needless  to  say,  the  list  of  information  requirements  is  far  from  all-inclusive.  It  is 
possible  that  many  important  topics  have  been  left  out.  The  blank  spaces  at  the 
end  of  the  section  are  provided  for  your  suggestions.  Please  be  as  imaginative  as 
you  can.  Your  new  and  innovative  ideas  will  be  incorporated  into  the  latter  stages 
of  this  project. 
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PLEASE  PROVIDE  YOUR  OPINION  AS  TO  HOW  MUCH  (OR  HOW  LITTLE)  THE 
AFAA  NEEDS  THE  FOLLOWING: 

1.  Access  to  current  regulations  and  directives  (AFRs,  DODIs,  AARs,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

2.  Access  to  obsolete  regulations  and  directives. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

3.  Access  to  technical  orders  (TOs). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

4.  Access  to  legal  references  (public  laws.  CG  decisions,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 
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5.  Access  to  “library-type"  references  (textbooks,  articles,  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  important 

5  -  Critical 

6.  Access  to  AFAA  audit  guides. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

7.  Access  to  AFAA-produced  audit  programs  (including  CDAPs). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

8.  Access  to  the  complete  text  of  AFAA  local  audit  reports. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 
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9.  Access  to  the  text  of  AFAA  “blue-book"  reports. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

10.  Access  to  data  related  to  AFAA  audit  reports  (includes  such  information  as 
number  of  findings,  report  title,  report  number,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

1 1 .  Access  to  non-AFAA  audit  guides,  programs,  and  so  forth  (i.e.,  DODIG,  GAO. 
and  AAA). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

12.  Access  to  non-AFAA  reports  of  audit. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 
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13.  Access  to  data  related  to  non-AFAA  audits  (report  titles,  number  of  findings 
etc.). 


Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

14.  Access  to  non-AFAA  information  systems. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

If  you  answered  “3."  “4."  or  “5"  to  question  #  1 4  and  know  of  any  non-AFAA  systems 
that  you  feel  auditors  need  access  to.  please  list  them  here  and  give  a  brief  synopsis 
of  the  value  of  accessing  the  system.  Use  additional  sheets  if  necessary. 
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15.  Access  to  classified  information  (consider  both  the  need  to  access  AFAA- 
generated  classified  information  [i.e.,  classified  audit  reports  or  workpapers]  and 
classified  information  belonging  to  third  parties  or  audit  clients). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

Questions  16-22  refer  to  your  need  for  data  during  project  management.  Your 
role  in  "project  management"  may  be  as  an  auditor,  audit  manager,  supervisor, 
executive,  or  audit  clerk.  Evaluate  your  need  for  the  following  types  of  information. 

16.  Access  to  information  about  ongoing  audit  projects  (i.e.,  date  started,  man- 
days  invested  to  date,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

17.  Access  to  information  about  planned  projects  (i.e.,  projected  start  date, 
planned  hours,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 
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18.  Access  to  summary  data  (i.e.,  data  that  is  “rolled  up“  or  summarized  by  a 
particular  office,  branch,  region,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

19.  Access  to  productivity  trends  (i.e.,  audit  starts  per  quarter,  direct  time 
percentages,  PMBs.  and  so  on)  in  each  office,  branch,  or  section. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

20.  Access  to  information  about  the  area  to  be  audited  (size,  mission,  when  last 
audited,  relative  Importance  of  the  area.  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 
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21 .  Access  to  information  about  nonconcurrences  (status,  date  of  report,  manage¬ 
ment  OPR.  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

22.  Access  to  information  about  potential  monetary  benefits  (date,  amount, 
amount  agreed  to  by  management,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 


Questions  23-25  refer  to  your  need  for  certain  types  of  nonaudit  information. 
(Keep  in  mind,  however,  that  Privacy  Act  information  is  carefully  controlled  and 
that  AFAA  cannot  set  up  any  unauthorized  systems  of  records.) 

23.  Access  to  information  needed  for  civilian  or  military  appraisals  (position 
description,  date  due.  employee’s  accomplishments  during  the  appraisal  period, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 


24.  Access  to  leave  and  other  timekeeping  Information. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 

25.  Access  to  employee  training  information  (dates  and  types  of  training  or 
education,  professional  certifications,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  No  value  at  all 

2  -  Some  negligible  value 

3  -  Nice  to  have 

4  -  Important 

5  -  Critical 


This  section  obviously  cannot  identify  all  of  the  many  types  of  information  that 
agency  personnel  need  to  do  their  jobs.  It  is  provided  so  that  you  can  identify 
information  requirements  that  you  feel  are  important.  Just  a  reminder,  however, 
we  are  trying  to  identify  requirements  for  information,  not  computers.  Please  try 
to  phrase  your  needs  in  terms  of  information  needed  (i.e. ,  “I  need  to  know  what 
suspenses  are  due  this  week.")  and  not  in  terms  of  hardware /software  (i.e..  “I  need 
a  faster  modem  and  a  WATS  line.").  The  hardware  and  software  required  to  meet 
your  information  needs  will  be  evaluated  later. 
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SECTION  3— ASSESSMENT  OF  INFORMATION  AVAILABILITY 


PURPOSE:  Section  2  identified  the  kinds  of  information  that  you  feel  are  needed 
to  do  your  job.  In  this  section  you  will  evaluate  how  well  these  informa  on  needs 
are  being  met.  After  each  statement,  please  circle  the  response  which  best 
describes  your  opinion.  Use  the  following  guide  to  determine  your  response: 

0 — No  opinion.  Circle  the  number  “0"  If  you  are  not  familiar  with  the  information 
described.  If  you  selected  response  “0"  to  the  corresponding  question  in  section 
2.  you  should  also  answer  “0”  here. 

1 —  This  information  is  not  needed  at  all.  Circle  the  number  “1"  if  you  feel  the 
information  described  is  not  necessary  to  the  accomplishment  of  the  audit  mission. 
If  you  selected  response  “1"  to  the  corresponding  question  in  section  2,  you  should 
also  answer  “1"  here. 

2 —  This  information  is  almost  impossible  to  get.  In  other  words,  it  would  require 
much  more  effort  than  the  Information  is  worth. 

3 —  This  information  is  available,  but  it  takes  a  lot  of  effort.  You  may  have  to  go 
outside  your  work  area  to  another  office  or  even  to  another  base.  You  may  also 
need  to  task  or  request  others  to  assist  in  the  effort.  When  you  are  finished,  you 
still  might  not  have  exactly  what  you  want. 

4 —  Ihis  information  is  available  with  a  reasonable  amount  of  effort.  You  may  have 
to  call  around  a  bit.  but  you  will  get  what  you  want  without  placing  an  un¬ 
reasonable  burden  on  anyone  else. 

5 —  7 his  information  is  readily  available.  However,  it  may  not  be  exactly  in  the 
format  you  want. 

6 —  Ihis  information  is  nearly  perfect.  It  is  in  ‘he  exact  formal  and  amount  of  detail 
that  you  want. 


The  blank  spaces  at  the  end  of  this  section  correspond  to  your  suggestions  from 
section  2.  Evaluate  how  well  each  of  your  requirements  is  currently  being  met.  If 
you  wish,  you  can  number  each  of  your  suggestions  and  use  the  number  rather 
than  recopying  the  entire  input.  If  you  elect  to  do  this,  however,  please  ensure 
that  (he  numbers  match  the  right  suggestions. 


76 


PLEASE  PROVIDE  YOUR  OPINION  AS  TO  HOW  WELL  (OR  HOW  POORLY)  EACH 
OF  THE  FOLLOWING  INFORMATION  REQUIREMENTS  IS  BEING  MET. 


1.  Access  to  current  regulations  and  directives  (AFRs,  DODIs,  AARs,  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

2.  Access  to  obsolete  regulations  and  directives. 

Circle  ONE  response. 

0  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

3.  Access  to  technical  orders  (TOs). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 
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4.  Access  to  legal  references  (public  laws,  CG  decisions,  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

5.  Access  to  “library-type"  references  (textbooks,  articles,  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  Impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

6.  Access  to  AFAA  audit  guides. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 
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7.  Access  to  AFAA-produced  audit  programs  (including  CDAPs). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

8.  Access  to  the  complete  text  of  AFAA  local  audit  reports. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

9.  Access  to  the  text  of  AFAA  “blue-book”  reports. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  gel 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 
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10.  Access  to  data  related  to  AFAA  audit  reports  (includes  such  information  as 
number  of  findings,  report  title,  report  number,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

1 1 .  Access  to  non-AFAA  audit  guides,  programs,  and  so  forth  (<  e..  DOD1G,  GAO, 
and  AAA). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

12.  Access  to  non-AFAA  reports  of  audit. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  Impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 
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13.  Access  to  data  related  to  non-AFAA  audits  (report  titles,  number  of  findings, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

14.  Access  to  non-AFAA  information  systems. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 


If  you  provided  some  responses  to  the  fill-in-the-blank  part  of  section  2  (non-AFAA 
systems  that  you  felt  auditors  need  access  to),  please  give  a  brief  synopsis  of  the 
ability  to  access  these  systems. 


15.  Access  to  classified  information. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

16.  Access  to  Information  about  ongoing  projects  (i.e.,  date  started,  man-days 
invested  to  date.  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

17.  Access  to  information  about  planned  projects  (i.e..  projected  start  date, 
planned  hours,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 
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18.  Access  to  summary  data  (i.e.,  data  that  is  “rolled  up"  by  office,  branch,  region, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  A 'most  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

19.  Access  to  trends  in  each  office,  branch,  or  section. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

20.  Access  to  information  about  the  area  to  be  audited  (size,  mission,  when  last 
audited,  relative  importance  of  the  area,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 


! 
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2 1 .  Access  to  information  about  nonconcurrences  (status,  date  of  report,  manage¬ 
ment  OPR,  etc.). 


Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

22.  Access  to  information  about  potential  monetary  benefits  (date,  amount, 
amount  agreed  to  by  management,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 


Questions  23-25  refer  to  your  need  for  certain  types  of  “nonaudit"  information. 


23.  Access  to  information  needed  for  civilian  or  military  appraisals  (position 
description,  date  due,  employee's  accomplishments  during  the  appraisal  period, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  Impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 
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24.  Access  io  leave  and  other  timekeeping  information. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

25.  Access  to  employee  training  information  (dates  and  types  of  training  or 
education,  professional  certifications,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Almost  impossible  to  get 

3  -  Available,  but  takes  a  lot  of  effort 

4  -  Available  with  a  reasonable  amount  of  effort 

5  -  Readily  available 

6  -  Nearly  perfect 

Ifyou  provided  any  additional  requirements  at  the  end  of  section  2,  please  assess 
the  availability  of  each  requirement.  Use  the  same  0-6  scale  that  you  used 
previously  to  fill  in  this  section.  You  need  not  recopy  every  entry,  but  please  be 
certain  that  the  reference  numbers  agree. 


Information 


Availability 


Information 


Availbility 
0  1  2  3  4  5  6 


0  1  2  3  4  5  6 


0  1  2  3  4  5  6 


0  1  2  3  4  5  6 


0  1  2  3  4  5  6 


0  1  2  3  4  5  6 


0  1  2  3  4  5  6 
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Availability 


J- 


0  12  3  4  5  6 


k. 


0  1  2  3  4  5  6 


1. 


0  12  3  4  5  6 


m. 


0  1  2  3  4  5  6 


n. 


0  1  2  3  4  5  6 


o. 


0  12  3  4  5  6 


i 


k 

P 


0  1  2  3  4  5  6 
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SECTION  4— ASSESSMENT  OF  INFORMATION  TIMELINESS 


PURPOSE:  Section  2  identified  the  kinds  of  information  that  you  felt  were  needed 
to  do  your  job  and  section  3  indicated  its  availability.  In  this  section  you  will 
evaluate  the  timeliness  of  the  information.  After  each  statement,  please  circle 
the  response  which  best  describes  your  opinion.  Use  the  following  guide  to 
determine  your  response. 

0 — No  opinion.  Circle  the  number  “0"  if  you  are  not  familiar  with  the  information 
described.  If  you  selected  response  “0  to  the  corresponding  questions  in  sections 
2  and  3.  you  should  also  answer  “0"  in  this  section. 

1 —  This  information  is  not  needed  at  all.  Circle  the  number  “1”  if  you  feel  the 
information  described  is  not  necessary  to  the  accomplishment  of  tht  audit  mission. 
If  you  selected  response  “1"  to  the  corresponding  questions  in  sections  2  and  3, 
you  should  also  answer  “1"  in  this  section. 

2 —  This  information  is  available  but  useless.  By  the  time  I  receive  the  i.  '  rmation, 
it  is  so  old  that  it  is  useless  to  me. 

3 —  This  information  is  available  and  has  some  value.  By  the  time  I  receive  the 
Information,  however,  it  is  somewhat  dated  and  of  limited  value. 

4—  This  information  is  generally  current  and  timely.  By  the  time  I  receive  the 
information,  however,  it  has  lost  some  of  its  value. 

5 —  This  information  is  current  and  on  time. 


The  blank  spaces  at  the  end  of  this  section  correspond  to  your  suggestions  from 
section  2.  Evaluate  how  weT  each  of  your  requirements  is  currently  being  met.  If 
you  wish,  you  can  number  each  of  your  suggestions  instead  of  recopying  the  entire 
input.  If  you  elect  to  do  this,  however,  please  ensure  that  the  numbers  match  your 
suggestions. 
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PLEASE  PROVIDE  YOUR  OPINION  AS  TO  THE  TIMELINESS  WITH  WHICH  EACH 
OF  THE  FOLLOWING  INFORMATION  REQUIREMENTS  IS  BEING  MET. 


1.  Access  to  current  regulations  and  directives  (AFRs,  DODIs.  AARs.  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

2.  Access  to  obsolete  regulations  and  directives. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

3.  Access  to  technical  orders  (TOs). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

4.  Access  to  legal  references  (public  laws,  CG  decisions,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 
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5.  Access  to  “library-type"  references  (textbooks,  articles,  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

6.  Access  to  AFAA  audit  guides. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  ail 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

7.  Access  to  AFAA-produced  audit  programs  (including  CDAPs). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

8.  Access  to  the  complete  text  of  AFAA  local  audit  reports. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  «nd  on  time 
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9.  Access  to  the  text  of  AFAA  “blue-book”  reports. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

10.  Access  to  data  related  to  AFAA  audit  reports  (includes  such  information  as 
number  of  findings,  report  title,  report  number,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

11.  Access  to  non-AFAA  audit  guides,  programs,  and  so  forth  (i.e.,  DODIG.  GAO. 
and  AAA). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

12.  Access  to  non-AFAA  reports  of  audit. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 
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13.  Access  to  data  related  to  non-AFAA  audits  (report  titles,  number  of  findings, 
etc.). 


Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

14.  Access  to  non-AFAA  information  systems. 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 


If  you  provided  a  response  to  the  flll-tn-the-blank  part  of  section  2  (any  non-AFAA 
systems  that  you  felt  auditors  need  access  to),  please  give  a  brief  synopsis  of  the 
ability  to  access  the  system  in  a  timely  manner.  Consider  the  entire  time  frame, 
including  any  preliminary  coordination  and  so  forth. 
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SYSTEM  NAME 


TIMELINESS 
(Rate  on  0-5  Scale) 


15.  Access  to  classified  information. 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 
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16.  Access  to  information  about  ongoing  projects  (i.e.,  date  started,  man-days 
invested  to  date.  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

17.  Access  to  information  about  planned  projects  (i.e.,  projected  start  date, 
planned  hours,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

18.  Access  to  summary  data  (i.e.,  data  that  is  “rolled  up"  by  office,  branch,  region, 
or  section). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  s^me  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

19.  Access  to  trends  in  each  office,  branch,  or  section. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 
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20.  Access  to  information  about  the  area  to  be  audited  (size,  mission,  when  last 
audited,  relative  Importance  of  the  area,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

21.  Access  to  information  about  audit  nonconcurrences  (status,  date  of  report, 
management  OPR,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

22.  Access  to  information  about  potential  monetary  benefits  (date,  amount, 
amount  agreed  to  by  management,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

23.  Access  to  information  for  civilian  or  military  appraisals  (position  description, 
date  due,  employee's  accomplishments  during  the  appraisal  period,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 
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24.  Access  to  leave  and  other  timekeeping  Information. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 

25.  Access  to  employee  training  Information  (dates  and  types  of  training  or 
education,  professional  certifications,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Available  but  useless 

3  -  Available  and  has  some  value 

4  -  Generally  current  and  timely 

5  -  Current  and  on  time 


If  you  provided  any  additional  requirements  at  the  end  of  section  2,  please  assess 
the  timeliness  of  each  requirement.  Use  the  same  0-5  scale  that  you  used 
previously  for  this  section.  You  need  not  recopy  every  entry,  but  please  be  c  nain 
that  the  reference  numbers  agree. 

Information  Timeliness 

a.  _  0  12  3  4  5 


b. 


0  1  2  3  4  5 
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Timeliness 


0  12  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 
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0  1  2  3  4  5 


0  1  2  3  4  5 
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SECTION  5— ASSESSMENT  OF  INFORMATION  FORMAT 


PURPOSE:  To  be  most  beneficial,  information  should  be  in  a  format  that  is  easy 
to  use  and  understand.  In  this  section  you  will  evaluate  the  format  of  the 
information.  After  each  statement,  please  circle  the  response  which  best  describes 
your  opinion.  Use  the  following  guide  to  determine  your  response. 

0 — No  opinion.  Circle  the  number  “O'  if  you  are  not  familiar  with  the  information 
described.  If  you  selected  response  “0"  to  the  corresponding  questions  in  sections 
2.  3.  or  4.  you  should  also  answer  “0"  in  this  section. 

1 —  This  information  is  not  needed  at  alL  Circle  the  number  “1"  if  you  feel  the 
information  described  is  not  necessary  to  the  accomplishment  of  the  audit  mission. 
If  you  selected  response  “1"  to  the  corresponding  questions  in  sections  2.  3,  and 
4,  you  should  also  answer  “1"  in  this  section. 

2 —  This  information  is  not  decipherable.  I  receive  the  information  in  a  form  that  is 
so  hard  to  understand  it  is  useless  to  me. 

3 —  This  information  is  hard  to  comprehend  or  use.  I  receive  the  information  in  a 
format  that  is  hard  to  use.  I  must  expend  a  great  deal  of  time  and  effort  to  put  the 
information  into  an  understandable  form. 

4 —  This  information  is  clear  and  easy  to  understand. 

5 —  This  information  is  in  the  exact  format  I  want.  If  I  decide  to  change  the  format, 
I  can  do  so. 


The  blank  spaces  at  the  end  of  this  section  correspond  to  your  suggestions  from 
section  2.  Evaluate  the  format  for  each  of  your  requirements.  If  you  wish,  you 
can  number  each  of  your  suggestions  instead  of  recopying  the  entire  input.  If  you 
elect  to  do  this,  however,  please  ensure  that  the  numbers  match  your  suggestions. 
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PLEASE  PROVIDE  YOUR  OPINION  ON  THE  FORMAT  OF  EACH  OF  THE  FOLLOW¬ 
ING  INFORMATION  REQUIREMENTS. 

1.  Access  to  current  regulations  and  directives  (AFRs,  DODIs.  AARs,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

2.  Access  to  obsolete  regulations  and  directives. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

3.  Access  to  technical  orders  CTOs). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

4.  Access  to  legal  references  (public  laws,  CG  decisions,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 
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5.  Access  io  “library-type”  references  (textbooks,  articles,  etc.). 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  1  want 

6.  Access  to  AFAA  audit  guides. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

7.  Access  to  AFAA-produced  audit  programs  (inclu 
Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

8.  Access  to  the  complete  text  of  AFy* 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehei 

4  -  Clear,  easy  to  mu' 

5  -  Exactly  the  form; 


9.  Access  to  the  text  of  AFAA  “blue-book"  reports. 
Circle  ONE  response. 


0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

10.  Access  to  data  related  to  AFAA  audit  reports  (includes  such  information  as 
number  of  findings,  report  title,  report  number,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  1  want 

1 1 .  Access  to  non-AFAA  audit  guides,  programs,  and  so  forth  (i.e.,  DODIG,  G40, 
and  AAA). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  1  want 

12.  Access  to  non-AFAA  reports  of  audit. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 
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13.  Access  to  data  related  to  non-AFAA  audits  (report  titles,  number  of  findings, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

14.  Access  to  non-AFAA  information  systems. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  1  want 


If  you  answered  the  fill-in-the-blank  part  in  section  2  (any  non-AFAA  systems  that 
you  felt  auditors  need  access  to),  please  describe  the  format  of  the  data  you  receive. 

SYSTEM  NAME  FORMAT 

(Rate  on  0-5  Scale) 
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15.  Access  to  classified  information. 


Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

16.  Access  to  information  about  ongoing  projects  (i.e..  date  started,  man-days 
invested  to  date,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

17.  Access  to  information  about  planned  projects  (i.e..  projected  start  date, 
planned  hours,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

18.  Access  to  suminaiy  data  (i.e..  data  that  is  “rolled  up”  by  office,  branch,  region, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 
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19.  Access  to  trends  In  each  office,  branch,  or  section. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

20.  Access  to  information  about  the  area  to  be  audited  (size,  mission,  when  last 
audited,  relative  importance  of  the  area,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

2 1 .  Access  to  information  about  nonconcurrences  (status,  date  of  report,  manage¬ 
ment  OPR,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  1  want 

22.  Access  to  information  about  potential  monetary  benefits  (date,  amount, 
amount  agreed  to  by  management,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 


23.  Access  to  Information  needed  for  civilian  or  military  appraisals  (position 
description,  date  due.  employee’s  accomplishments  during  the  appraisal  period, 
etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  1  want 

24.  Access  to  leave  and  other  timekeeping  information. 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 

25.  Access  to  employee  training  information  (dates  and  types  of  training  or 
education,  professional  certifications,  etc.). 

Circle  ONE  response. 

0  -  No  opinion  expressed 

1  -  Not  needed  at  all 

2  -  Cannot  decipher 

3  -  Hard  to  comprehend  or  use 

4  -  Clear,  easy  to  understand 

5  -  Exactly  the  format  I  want 
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If  you  provided  any  additional  requirements  at  the  end  of  section  2,  please  assess 
the  format  of  each  requirement.  Use  the  same  0-5  scale  that  you  used  previously 
for  this  section.  You  need  not  recopy  eveiy  entry,  but  please  be  certain  that  the 
reference  numbers  agree. 

Information  Format 

a.  _  0  1  2  3  4  5 


b. 


0  12  3  4  5 


c. 


0  1  2  3  4  5 


d. 


0  1  2  3  4  5 


e. 


0  1  2  3  4  5 
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Information 


Format 


0  1  2  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 


0  1  2  3  4  5 


Information 


Format 


1. 


0  12  3  4  5 


m. 


0  1  2  3  4  5 


n. 


0  1  2  3  4  5 


o. 


0  12  3  4  5 


P- 


0  1  2  3  4  5 
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